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A preliminary draft of this report was prepared by Mr. Leander Smith 
in the spring of 1964. Mr. Smith had been the project coordinator for the 
SMSG Programed Learning Project since its inception in the spring of 1961. 

The statistical analyses contained in the report were carried out by 
Mr. Ronald Pyszka, Research Assistant at SMSG headquarters, during the 
1964-65 academic year. Mr. Pyszka prepared a second draft of the report 
which incorporated the results of the statistical analysis. 

The final draft of this report was prepared by Mr. William Chinn, 
also a member of the SMSG headquarters staff, as well as a participant 
in the writing team which prepared the programed text. This final draft, 
prepared in the fall of 1965, incorporated comments and suggestions made 
by members of the SMSG Programed Learning Panel in response to the 
second draft of the report. 
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Recommendations for the Study 

Following B. F. Skinner’s articles, "The Science of Learning and the Art 
1 2 

of Teaching" in 1954 and "Teaching Machines" in 1958, there appeared many 
programed, texts with subject matter ranging from statistics to foreign language 
training. Extensive programing of mathematics by persons expert in programing 
but not in mathematics has led the SMSG Advisory Board to decide that a careful 
study of programed learning should be undertaken with specific reference to SMSG 
materials. It was felt that although programed, instruction had evolved as a 
psychologist’s tool, the SMSG efforts should be carried out jointly by mathe- 
maticians, teachers of mathematics, and psychologists under the supervision of 
a Panel on Programed Learning. 

In 1961, an ad hoc committee on programed learning of mathematics was ap- 
pointed to advise the Director of SMSG on procedures for carrying out the de- 
cision of the Advisory Board for a careful study. The committee met at Harvard 
University on May 2, 1961 . It was agreed, by this committee that, although the 
SMSG 7th grade course would be technically easier to program, an attempt to 
program the SMSG 9th grade course would, provide more useful information. The 
following recommendations were made: 

1. Mathematicians from both high school and university levels should be 
taught to construct programs. 

2 . The Director of SMSG should locate a suitable number of qualified in- 
dividuals who could devote part of their time during the 1961-62 academic year 
to the project. 

3 . Each section of the SMSG First Course in Algebra should be translated 
into programed form. 

4. A variety of programs should be prepared. 

^ Lumsdaine, A. A., R. Glaser, Teaching Machines and Programed Learning : A 

Source Book , DAUI of NEA, Washington D.C. , i 960 , pp. 94-113. 

2 , op. cit., pp. 137-72* 
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5. The program "writers should retain the content and sequence of the SMSG 
First Course in Algebra in order to make comparisons possible. 

6. The problem of motivation should be kept in mind in writing the program. 

7. Feedback should be obtained for the various drafts from students who 
try the program. 

8. The objective of the first part of this SMSG study should be to verify 

the hypothesis: that the SMSG material can be presented as effectively according 

to prescribed criteria, through programs as through conventional classroom- 
textbook procedures. 

9. Evaluation of any program should involve the use of tests which require 
students to solve problems and recognize concepts outside the text. 

The translation of the SMSG First Course Algebra was to be accomplished 
by a small group consisting of a university mathematician and classroom teachers 
thoroughly familiar with the course. The variety of programs should include a 
constructed response program (featuring small steps at low error rate with some 
branching and occasional inclusion of large steps as variations to be experimented, 
with), a multiple- choice program (featuring a scrambled book format with expla- 
nations of incorrect answers using the size of the unit as a variable to be 
experimented with), and a section (sections) to be used as supplement(s) to a 
textbook. With reference to the problem of motivation, the program writers were 
to take a close look at what the classroom teachers do to supplement the motiva- 
tion supplied by the text. For feedback on the various drafts, it was recommend.ed 
that the first draft be used to obtain detailed comments and information from a 
small number of students, involving the possible use of machines. 




The SMSG Panel on Programed Learning (June 1961) 

Following the recommendations of the preliminary committee, the Director 
appointed a Panel on Programed Learning consisting of R. C. Buck, University of 
Wisconsin; E. E. Hammond, Jr., Philips Andover Academy; L. D. Hawkinson, San 
Francisco Unified School District; J. G. Holland, Harvard. University, W. J. 
McKeachie, University of Michigan; E. E. Moise, Harvard University; H. 0 . Poliak, 
Bell Telephone Laboratories; and D. W. Taylor, Yale University. 

The first meeting of the SMSG Panel on Programed Learning was held at 
Har vard University on June l6, 1961. It was agreed that a committee consisting 
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of V. H. Haag, H. 0 . Poliak, and C. E. Rickart (members of the writing team on 
the SMSG First Course iri Algebra ) , should prepare a chapter summary and statement 
of objectives for the algebra. 

Other specific operational recommendations of the Panel include the following: 

1 . Program items, written on cards with answers on the back should, be tried 
on a small number of students to detect gross errors, ambiguities, etc., and re- 
vised on the basis of student feedback. 

2 . The program should be reviewed by mathematicians, teachers of mathematics, 
and psychologists for both quality of content and quality of programing. Then, 

a revision should be prepared. 

3. Additional motivational material may be added if the same material is 
made available to classes using the conventional text during the experimentation 
in 1962-63. 

The Manual for Programer s (July 196 l) 

Various stages for the implementation of the project include: the production 

of programed materials, trial runs with groups of students, evaluation of results, 
revisions, and further study. 

Preliminary to the full scale production of materials, a Manual for Programers 
was drafted by the Coordinator of the project for use in a summer workshop and by 
the writing teams. The manual was intended as a basic document to provide persons 
having a thorough knowledge of the mathematics with information that they would, 
need for the acxual construction of programs. It included background, material on 
the psychology as described by B. F. Skinner, an overview of the state of the art 
of programing, specific recommendations for the analysis of the content of the 
SMSG First Course in Algebra , and notes on the construction of items. 

After the draft manual was checked by members of the Panel for glaring errors 
in content, it was then used by the Workshop in August 1961 and revised for use by 
the writing teams during I96I-62. As more knowledge was gained about programing, 
materials were added to or deleted from the Manual (see APPENDIX F for revised 
manual) . 

The Workshop (August 1961) 

During the summer of 1961, the Director of SMSG and the Project Coordinator 
located sufficient manpower to establish twelve writing centers to work on a 
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part-time basis during the 1961-62 academic year. Prom each of these centers, 
representatives were assembled in New Haven, where the 1961 SMSG Summer Writing 
Session was being held, for a workshop from August 7 to l8. The following 
people attended the Workshop: D. Blakeslee, San Francisco; M. P. Bridgess, 

Boston; F. Jacobson, New Haven; H. Jones, Stillwater; W. Matson, Portland, Oregon; 
0. Peterson, Emporia, Kansas; P. Redgrave, Norwich Connecticut; W. Storer* 

Des Moines; and. H. Swain, Winnetka. In addition to the participants from writing 
centers, SMSG secured J. C. Hammock, a psychologist at Bell Telephone Laboratories 
and P. I. Jacobs, a psychologist from Educational Testing Service. 

The purposes of the Workshop were; 

1. to acquaint a core of writers with programed instructional materials 
and. the procedures for producing them; 

2. to prepare sample units in constructed response and multiple-choice 
formats which could serve as models for the center writing teams; 

3. to take on writing assignments and suggestions for the organization of 
the twelve centers. 

To accomplish the first purpose, the draft of the Manual for Programers was 
examined, criticized, and. used as the basis for writing sample units. In pre- 
paring sample units, Sections 1-1 and. 6-5 of the SMSG First Course in Algebra 
were selected for the programing in the Workshop. Participants formed two writing 
teams; one to produce a constructed response program, the other to produce a 
multiple- choice program. With the experience of writing as a- team in New Haven, 
the participants were then able to anticipate problems of producing programs and 
orienting other members of the centers. Lumsdaine and Glaser’s Source Book and 
Keller s Reini or cemen t Theory provided psychological background on programing; 
literature produced by the Center for Programed Instruction (N.Y.C.), Doubleday 
Tutor- Texts, TEMAC, and. Holland and Skinner’s Analysis of Behavior 2 provided 
examples of existing programs. 

It is important to note that in this phase of the Project, an attempt was 
made to produce two "pure form" programs. Each program was to consist of either 



Keller, F. S., Learning: Reinforcement Theory , Random House, New York, 1954. 

Holland, J. G. , B. F. Skinner, Analysis of Behavior , McGraw-Hill Inc., New 
York, 1961. 






1 



i 



'J 



< 



i 



1 



i 



'i 

1 



\ 






<i 






all constructed response or all multiple- choice items. The order of topics and 
the point of view of the SMSG First Course in Algebra were to be preserved. 

The Writing Centers (Academic Year 1961 - 62 ) 

Each of the twelve writing centers consisted of three or four mathematicians 
and/or mathematics teachers whose acquaintance with SMSG First Course in Algebra 
stems from participation in the original writing team, the evaluation centers, 
or teaching SMSG mathematics to teachers. With knowledge of the mathematics 
well in hand, it was felt that these people could be provided with the infor- 
mation that they would need for the construction of programs. The centers were 
located in Boston, Chicago, Des Moines, Snporia (Kansas), New Haven, Palo Alto, 
Portland (Oregon), San Francisco, and. Stillwater (Oklahoma). Three centers were 
located in Chicago, two in Palo Alto, and one in each of the other cities named. 
The sections of the First Course were partitioned among the centers for programing. 

Twelve critics were selected to review the writing done in the centers^ six 
to review the constructed response materials and six to review the multiple- choice 
materials. Each reviewer was to comment specifically on the frames and to react 
generally to the section, using the following guide: 



Check on the scale where you feel this unit ranks. 
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Unit Panel Use: 
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Other comments on the unit or subunit 



Mode of Operation 

To show clearly the operation of the writing centers during the academic 
year 1961-62, it is best to take a representative center and di'cuss its mode 
of operation. Each center consisted of one mathematician and three teachers of 
mathematics operating on a part-time (2 days per month) basis. The mathemati- 
cian served in each case as a consultant on the mathematics content and as the 
final editor of the material. Each member of the writing team first generated 
an outline of the sections to which he was assigned. This outline was based on 
the content and organization of the SMSG First Course in Algebra and the Summary 
and Objectives outlined by Haag, Rickart, and Poliak (See APPENDIX E and 
APPENDIX 4 in Manual p. 9l). The problem set of each section was surveyed 
for problems which occurred early in the textual material. A first draft was 
then prepared on file cards. 

In the case of the constructed response program, the correct response was 
placed on the back of the card} in the case of the multiple- choice program, the 
alternatives were written on subsequent cards (usually of different color for 
ease in identifying). The first draft was then passed to other members of the 
center and to the center chairman for suggestions. On the basis of these comments 
the original writer prepared a second, draft version. Next, the revised draft 
passed to the center chairman for editing, who, in turn sent the draft to SMSG 
(then at Stanford University) for ditto reproduction. 

Each reviewer then commented on the particular phase of the work in which 
he was best fitted: the psychologist commented on the programing per se, the 

mathematician on the correctness of the presentation of the mathematics contained 
in the First Course , and the teachers on the appropriateness of language to the 
high school student. All the reviewers’ comments were returned to Stanford where 
they were consolidated by a typist and. sent back to the writing, team as guide- 
lines for further revision. 
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On receipt of the comments from the reviewers, the writing team reconsid- 
ered the language, mathematics, and programing technique, and prepared another 
draft which was forwarded to SMSG for lithographing. 

Experimental Centers (Spring 1962) 

Once the lithographed version of the first eight chapters of the Programed 
First Course in Algebra was produced (January 1962), classes were sought to try 
out these materials (to start in March 1962). It was suggested that eighth 
grade students, ready to begin algebra would be appropriate students. Other 
classes were to include: slow ninth grade students, students in upper grades 

who had not taken algebra, or repeating students whose earlier experience was ^ 
not with SMSG- like mathematics. 

A two part pre-test was designed to serve as an inventory of skills and 
concepts already in the command, of the students. The first part surveyed ma- 
terial covered in the 7th and 8th grade SMSG classes; the second part was 
to be used as both pre and post tests. 

In this pilot study, data collection and summarization was not adequate. 
Teachers* summaries of student results were sketchy, at best, and student re- 
sponse sheets themselves were too voluminous to enable summarization in time 
for the 1962 Summer Writing Session. The pre and post tests, however, were 
useful in the generation of revised testing (PLP-2 and PLP-l). Teachers 
indicated too, that chapter tests would also be a valuable adjunct to the program. 

The Summer Writing Session (Simmer 1962) 

The 1962 Summer Writing Session participants were selected, on the basis 
of demonstrated ability in writing programed material and in meeting deadlines; 

> role determination based on the need for a psychologist, mathematicians, and. 

teachers; perception of the problem and goals of SMSG; and freedom to devote 
eight weeks of their time to this project. The participants were: John D. Baum, 

David W. Blakeslee, M. Philbrick Bridgess, Mrs. Marjorie French, James E. Gilbert, 
I Arthur A. Hiatt, Stephen Hoffman, Mrs. Margaret Matchett, William W. Matson, 

1 Mrs. Persis 0. Redgrave, Winfield Roberson, Robert E. Schweiker, William Storer, 

f\ George Truscott, and Mrs. Helen Wehe. The participants were subdivided into 

two writing teams: one to revise the constructed response program, and the other 

to revise the multiple- choice program. Mr. Bridgess chaired the multiple- choice 
(Form MC) team; Mr. Blakeslee chaired the constructed response (Form CR) team. 
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It had become clear during the course of the year when the writing was done 
at the twelve individual centers ( 1961-62), that the ability of each writer to 
reproduce the material of the course in a language for 9th grade students to 
read was widely diverse. It had also become clear, as the writing progressed, 
that the idiosyncratic behavior of the individual writers was to appear in the 
Programed First Course in Algebra as writers sought to elaborate on material 
treated briefly in the original text, to create proofs where none had appeared, 
and to replace problems with their own versions of "good items to ask". In the 
summer writing sessions where two editors passed judgment on the content, there 
was more uniformity in format, and the divergence from the original text was 
reduced — in keeping with the stated objectives of this study. 

Since the student response sheets from the spring try-out had not been pro- 
cessed by the time the writing teams assembled on June 22, 1962, the writers set 
aside student try-out materials and worked from the lithographed version as a 
first draft. Although the twelve widely separated centers had spent most of 
their time on the first eight chapters and had left Chapters 9-17 in rough 
ditto form, the 1961-62 efforts still represented a real starting point; their 
efforts enabled the summer writers to generate materials for use in schools in 
the 1962-63 academic year. 

With each of the two teams working during the Summer Session, considerable 
time was spent at the beginning of the session on style, format, semantics, and. 
the more basic problem of screening the materials produced by the center writing 
teams. The mathematicians on the summer writing team began by preparing a first 
version of the lithographed version of the course. In some cases, this necessi- 
tated re-examination of the First Course in Algebra and re-evaluation of the 
goals of that course. Some of the areas that presented challenge were in the 
programing of problem sets, proofs, and review topics. In addition, each team 

encountered problems that might be considered characteristic of that particular 
format. 

The writers of the multiple- choice program spent considerable time antici- 
pating distractors (alternatives which require careful discrimination and know- 
ledge of skills and/or concepts). The multiple-choice format enabled some 
preservation of text but suffered in its ability to elicit computational practice 
because of the form of presentation. The multiple-choice program that was pro- 
duced that summer, although a scrambled book in form, was essentially a single- 
track program through which a student would proceed. Very little effort was 
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made to capitalize on the multiple-track program that might he possible in 
later editions, given more time to write. 

The writers of the constructed response program, on the other hand, found 
the fragmentation of the earlier version quite a problem. There was an initial 
tendency to use lengthy sets of problems for practice, overcuing, and little 
synthetic material. Finding a need for synthesis, the writers redefined ‘'frame 1 '. 

A "frame" had been conceived as a short passage, three to five lines, with a 
significant word or words missing. The response sought was then to contribute 
to the learning of the skill or concept. Each frame was then defined as a "bit" 
or "response eliciting item". 

As the summer progressed, the fragmentation of content into lengthy sequences 
of "bits" began to draw reaction from the writers. They began to group exercises 
of similar nature into a box or frame; they then sought to use the "frame" as a 
means of identifying a recognizable step in the learning process. Each frame 
evolved as a conceptual unit; that is, an identifiable step toward the mastery 
of a skill or concept. By having the manuscript passed among each of the writers, 
the changes in format were quickly assimilated and further suggestions resulted. 

The ease of programing seemed to depend on the explicit structure of the 
content presented in the text and the independence of topics being developed. 

It was relatively easy to program Section 1-1 (Sets and Subsets) where the 
content prescribes an order of presentation. On the other hand, it was very 
difficult to program Section 1-2 (Sets and the Number Line) where the content 
of the original text did not suggest an orderly presentation and the number of 
topics which might be included had to be considered. Chapter 7 (Properties of 
Multiplication) and Chapter 12 (Polynomial and Rational Expressions) were most 
difficult to program. As mentioned before, programing of proofs was a challenge. 
To overcome part of the difficulty in programing proofs, use was made of the 
constructed panel in the CR format. Here, the student was led to complete 
steps (or to supply reasons for steps) in a proof, transferring these responses 
to appropriate blanks in a Panel to complete the proof. The constructed proof 
was then compared with a model proof as it might appear. Thus, the Panel con- 
struction served to integrate the small pieces of the proof into a whole. Also, 
in the process of reconstruction, more than one pass was made in the proof. 

The results of the 1962 Summer Writing Session were three-fold: 

1. The SMSG Programed First Course in Algebra, Form CR: a constructed 

response programed text with 13,552 responses in a total of 1,712 pages. All 
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items were written on right hand pages only to allow a right hand slider to he 
used to conceal confirmations; then, having gone through the hook using right 
hand pages, the student flips the hook over and again used right hand pages 
(the left hand pages when the hook is in normal position). Form CR was 
’’perfect hound” in six volumes. One of the problems caused hy the l 80 ° ro- 
tation of the hook had to do with reference to various Panels which had been 
printed in the hack sections of the hook when it is normal position. After the 
flipping, the Panels appeared in an upside down position. 

2 . The SMSG Programed First Course in Algebra , Form MC: a multiple- choice 

response programed text with about 2,200 items in a total of 2,357 pages hound 

in six volumes. The hooks were scrambled within sections; that is, each section 
began in position 1A and was paged independently from the rest of the text. 

A student’s path through a section might thus read: 1A, 10 C, kk, 9 B, IB, 11 A, 

8A, 6C, 2 D, I3D, 5 B, 11 B, 7 C, 10 B, 12 B, 3D (Section 1 - 1 , Form MC), according 
to the response he gives to each item. 

3. The PIP Chapter Tests: a series of tests covering the content of the 
17 chapters of the Fir st Course in Algebra ; also, pre and post tests based on 
the preliminary versions written during the 1961-62 academic year. The pre and 
post tests were designed to cover the following material: 

PIP -1 — inventory of 7th and 8th grade mathematics; 

PIP- 2 — first semester algebra; 

PIP-3 -- second semester algebra with overlap. 

Test Teaching the Preliminary Editions (1962-63) 

In the spring of 1962, a design was planned for experimentation during the 
1962-63 academic year. Schools throughout the country were asked if they would 
he interested in test-teaching the SMSG materials. These schools were selected 
on the basis of earlier participation in SMSG test- teaching of the First Coarse 
in Algebra and on the recommendations of members of the writing team. 

The design called for testing the following four modes of instruction: 

MC - the multiple- choice programed, text; 

CR - the constructed response programed text; 

MCR - the constructed, response machine format (unbound CR in Koneept- 0 - 
Graph K 0 G- 7 ); 

9 F - standard. SMSG First Course in Algebra . 



ERIC 

hfliflaffHHaoaa 



10 



Each mode was to he used under certain of the following conditions: 

Time-paced with a teacher: 

assigned with a due time or due date in blocks or sections with a 
teacher to control disbursement and to answer questions and discuss or 
explain any mathematics. 

Time-paced with a monitor: 

assigned with a due time or due date in blocks or sections with a 
monitor to control disbursement and to answer nonmathematical questions. 

Self-paced with a teacher: 

in classroom or home with a teacher to record time and to answer ques- 
tions and. discuss or explain any mathematics. 

Self-paced with a monitor: 

in classroom or library with a monitor to record time and to answer 
nonmathematical questions. 

Students were to be regular ninth grade algebra students, accelerated eighth 
grade students, or remedial tenth grade students. Teachers were asked to de- 
signate the ability level of their classes, but no standardized testing was 
administered for this purpose. All students were given pre-tests (PLP-1 and 
PLP-2), the seventeen chapter tests, and post- tests (PLP-2 and PLP-3). All 
student test answer sheets were scored and returned to SMSG for analysis. 

The response of the schools was generally very good. As it turned out, 
however, teachers used the statement of the project coordinator, "If for some 
reason, students fail to benefit from the program, SMSG should be contacted 
and. action taken to save the students." as an escape, a method of reverting to 
"normal" classroom instruction. Often, the notion of self-paced classes was so 
aversive to teachers that they established "minimum" rates which, in turn, 
became expectancy levels. Almost all self-paced classes reverted to some form 
of time-pacing at some, point during the year. Since they thus constituted a 
conglomeration of methods of presentation rather than pure form of self-pacing, 
they were eliminated from the analysis. 

The four treatment groups chosen for analysis were MC (multiple-choice), 

CR (constructed response), MCR (constructed response, machine format), and 
9F (standard SMSG algebra test) — all time- paced at 9th grade. The number of 
students in each mode is given in TABLE NUMBER 1 . 
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TABLE NUMBER 1 



Mode 


N 


CR 


260 


MC 


3^1 


MCR 


142 


9F 


138 



The MCR classes were supervised by monitors while all others were supervised 
by teachers. It is believed, though, that there is no difference between the 
way the monitors handled their classes and the way the teachers handled theirs. 

Since the four treatment groups differed on PLP-1 (pre) and. PLP-2 (pre) 
(measures of initial level of achievement), a covariance technique was used to 
give estimates of what the scores on PLP-2 (post) and PLP-3 (post) would have 
been had the four treatment groups been comparable on the two pre-tests. These 
estimates, or adjusted means, were tested for significance of difference among 
treatment groups. (See APPENDIX A.) 

The analysis was done twice, once with PLP-2 (post) as the criterion vari- 
able (adjusting for differences on PLP-1 (pre) and PLP-2 (pre)) and once with 
PLP-3 (post) as the criterion variable (adjusting for differences on PLP-1 (pre) 
and PLP-2 (pre)). With PLP-2 (post) as the criterion variable, the analysis 
indicated that both the Constructed Response and the Multiple -Choice modes are 
significantly better than 9F while there is no significant difference between 
them (all significance is at least the .05 level). Also, both CR and MC 
are significantly better than MCR while there is no significant difference 
between MCR and 9F. With PLP-3 (post) as the criterion variable, the results 
were identical except that the difference between CR and 9 F was not significant. 

In interpreting the results of those comparisons involving MCR, it is 
necessary to recall that the classes using this mode were administered by moni- 
tors while the classes using the other modes were administered by teachers. It 
is impossible to say what role this teacher-monitor variable plays in the ob- 
served differences between MCR and. the remaining three modes. 

In TABLE NUMBER 2 the adjusted means for PLP-2 (post) and. PLP-3 (post) 
are presented. In addition, the mean differences between PLP-2 (pre) and 
PLP-2 (post), adjusted for differences on PLP-1 (pre), are presented. It is 
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evident that, while many of the differences among treatment groups are signifi- 
cant, they are small in amount. 



TABLE NUMBER 2 



Mode 


Adjusted Means 


PLP-2 (diff) 


PLP -2 (post) 


PLP -3 (post) 


CR 


9.60 


l 6.66 


10.72 


MC 


9.16 


16.22 


10.93 


MCR 


6.23 


13.86 


9.28 


9F 


7.48 


14.90 


9.89 



Teacher and. Student Reactions ( 1962 - 63 ) 

There was intense negative reaction from some of the teachers and students 
against the boredom they found resulting from the programs. Much of this re- 
action appeared to be generated by the sameness of format. In the case of the 
constructed response program, boredom was intensified among the faster students 
who were forced to go through long series of items designed, primarily to teach 
the slower students. In the multiple-choice program, which provides for branch- 
ing and. skipping to care for the individual differences, there was objection to 
the "page-flipping” involved in the scrambled book technique; they also objected 
to being ■unable to review systematically without retracing all the steps in the 
program. In both cases, frustration was heightened by the books falling apart. 

In February 1963 , a questionnaire was sent to all 150 teachers in the 
Programed. Learning Project to gain insights on what future action SMSG should 
take in the area of programed learning. From these, l4l responded, with the 
following sorts of information: 

Number of years experience teaching SMSG Algebra : 

(0 years - 35 teachers, 1 - 42, 2 - 21, 3 - 23 , 4 - ll) 

Proportionately, the greatest support for programed learning came from those 
teachers with four years of experience with the algebra: 8 /ll as opposed, to 

the overall total of 73/132. Of the 140 teachers reporting, 79 favored 
programed learning, 6l were opposed. Teachers of the eighth, eleventh and 
twelfth graders favored the program most; in fact, teachers in 15 of the 
l8 eighth grade high-ability groups would use the material again. Favorable 









reactions were reported in 34 of the 63 schools using Form CR and in 28 
of the 4-2 schools using Form MC. The machine users were negative in a ratio 
of 2 to 3. 

Some specific comments by a number of teachers of Form CR are listed be- 
low. Relating to: writing style — "Text is too wordy", "The program appears too 

easy", "The students feel lost in too many frames"; form of presentation — "Most 
students tend to move too slowly when allowed to set their own pace", "Self- 
paced instruction does not seem appropriate for eighth grade students"; personal 
reaction — "At the moment, I feel so negative about the material that I could not 
be objective"; observation of student response — "...most students are happy to 
know exactly and instantly when they are wrong", "It is too easy to cheat." 
content — "There appears a lack of overview of subject matter"; accompanying 
tests— "The tests are too hard", "The tendency to do work mentally handicaps 
them when they get to tests where the steps are not outlined for them"; overall 
weakness — "Most of the weaknesses of the programed text are the weaknesses of the 
original text"; and suggested modifications — "Frames should be grouped by single 
ideas", "I suggest some flexibility to allow teachers to choose certain addi- 
tional or fewer exercises dictated by student need", "More problems need to be 
given after the material is presented step by step", "The program needs an 
index", "Answer sheet could be arranged better to facilitate grading". 

Some criticisms from teachers using Form MC are listed below. Relating 
to: student response- -"Students admit they are learning but feel it is more 

work than would be necessary in a regular class. They find it difficult to re- 
view for a test", "The better students seem to like the MC format better than 
classes using the CR", "Students are bored; they need more problem-solving"; 
presentation — "Program was exciting initially. It needs variety and possibly 
additional problem sets", "Good readers seem to do well; poor readers just 
can’t seem to get anywhere", "Program is boring. The chapters would be better 
partially programed so the teacher could teach and use the program as homework"; 
shaping of learning — "There is no place where the student learns to put a pro- 
blem in good form and. arrange his work in an efficient manner". 

From teachers who used teaching machines (Koneeptograph, KOG-7) the reaction 
was almost universally: 

"The machines are not satisfactory at all- -about half are now (January 1963 ) 
’unusable because of mechanical deficiencies." As a result, SMSG recalled the 



14 



machines and destroyed all hut a few which were retained for experimentation on 
controlled testing. The program used in the machines was Form CR in an unbound 
format to enable single sheet-fed operation. 



The most often expressed comments on the questionnaire about the program 
dealt with the presentation and format. Both students and teachers missed class- 
room discussions, without which they were unsure that progress was being made. 

The students wanted, to hear the material summarized and "pulled together". The 
students liked to know when they were wrong and why the error had been made. On 
the matter of format, they claimed the presentation was not sufficiently varied. 



The Hybrid. (Spring - Summer 1963) 

From the widespread dissatisfaction of teachers and students in the 1962- 
63 classes came the feeling that the "pure form" programs had. made SMSG mathe- 
matics less palatable than the text on which they were based. Hence, in the 
spring of 1963 the Project Coordinator prepared, a sample which combined the 
features of both earlier versions into a hybrid program. The features which 
were felt essential included: 

(1) immediate confirmation on most constructed response items ; 

(2) confirmation and correction of errors on multiple- choice items without 
scrambling the text; 

(3) inclusion of conventional textual material; 

(!{.) inclusion of reviews and problem sets with confirmations given in the 
back of the book; 

(5) inclusion of optional sections for more able students and. for students 
who want or need additional help; 

(6) inclusion of index, table of contents, tailor-made response sheets; 

(7) variation in format to relieve boredom; 

(8) provision for skipping items by those who have successfully completed 
criterion items. 

With the sample hybrid format and experienced writers of constructed response 
and multiple- choice programs, the 1963 SMSG Summer Writing Session began. 

Mrs. Persis Redgrave was asked to chair and. edit the work of the writing team 
consisting of D. Blakeslee, M. P. Bridgess, F. Elder, J. C. Hammock, F. Jacobson, 
M. Matchett, W. Matson, L. W. Smith, W. Storer, M. Wecheler, and M. Zelinka. 
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During late spring 1963, three classes were chosen from classes using each 
of the three texts (9P, MC, and CR). The students in these classes were ninth 
graders whose mean scores on PtP-1 were approximately equal: CR, 11.47; MC, 

10 -93; 9F> 11.11; total, 11.17. Item analysis of the chapter tests for these 

students was then done to provide a guide for the 1963 Summer Writing Team. 

In addition, one in every eight students using mode CR and mode MC in the 
1962-63 testing was required, to return his response sheets. For students 
using mode CR, a summary of the number of errors made on each item in the pro- 
gram, together with the total number of attempts^ was prepared. 

Since existing programs in mathematics had tended to be reproductions of 
the work of Skinner (CR) and Crowder (MC), there were no printed models of a 
hybrid for the writing team to begin with. Hence, as soon as Section 1-1 had 
been written, ’’sample pages” were set up by the typists and printed to show what 
the text would look like when finished. A fine gray screen was laid over the 
confirmation areas and. all items requiring a response were framed. The gray 
zone identified material to be red after students had made their responses to 
specific items. Tailor-made response sheets were prepared, to be printed, bound 
into the back of each volume, perforated for students '-.o remove. 

After a few sections of Form H (the hybrid format) had been written, a 
student was found, to go through the program. Having studied no algebra previously, 
he worked under the surveillance of Dr. Hammock who recorded the time spent, checked 
over the errors (many of them typographical) and administered the testing. In 
39*6 hours, the student was able to complete the first eight chapters, working 
between two and. three hours daily on the program and stopping between sections 
and at the end. of the chapters to take tests. Since this represents about half 
of the course, it was estimated that, if students worked on a self-paced basis, 
they would complete the 17 chapters in approximately 90-100 working hours. 

At the end of eight weeks of intensive writing, SMSG Programed First Course 
in Algebra (Form H) emerged in preliminary form with 1,036 pages, plus key, 

index, response sheets, and a total of 8,782 items to which the student should 
respond. 

In addition to the preparation of Form H, a form was developed for comple- 
tion by the teachers for their chapter by chapter evaluation, their evaluation of 
student response to the program with reference to their ability levels, the es- 
timated. time for completion of each section, areas of difficulty encountered, and 
recommended changes. The 1963 writers also attacked the problem of testing 
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with new vigor. The 1962-63 tests had been criticized by teachers as being too 
hard, unfair samples of content, and poorly worded. An experienced writer-teacher, 
Florence Elder, was selected to rewrite the ter ‘.s for 1963 - 64 . Items were sub- 
mitted by each of the programers, worked into a 35 minute test, checked by the 
writers again, and administered to a student subject. Th" feeling among the 
writers was that these new tests represent a more characteristic sample of the 
skills, concepts, and vocabulary on a chapter by chapter basis than the earlier 
tests. 

PLP Centers (1963-64) 

In 1963-64, an attempt was made to evaluate Form H in a large scale ex- 
periment with Forms CR, MC, 9 F, and the SMSG Introduction to Algebra using common 
testing on all groups. The Introduction to Algebra text (IA) covers essentially ' 
the same material as the First Course in Algebra ( 9 F), with simplified language, 
slower pacing, and more extensive illustrations. 

Schools to participate in 1963-64 were selected from those which had parti- 
cipated in 1962-63 and indicated they were interested in continuing, and other 
schools which had expressed a desire to take part in this SMSG Project. The 
teachers and students were assigned by the schools to the texts provided by SMSG. 
The conditions imposed by SMSG were that: 

1 . all teachers should have had at least one year experience with SMSG 
Algebra, 

2 . all students were to be in the upper three quartiles on SCAT 2A (School 
and College Ability Test, Form 2 A) to be administered in September 1963, 
and 

3. teachers who were to teach Form CR or Form MC in 1963-64 must have 
taught these forms in 1962-63. 

The first cr-. iition was to assure trained teachers, familiar with the content of 
SMSG Algebra , to preclude philosophical disagreement with the subject. The second 
condition was to assure SMSG that the students were selected from those who would 
normally take algebra in a precollege course. The third condition was imposed to 
compare teachers using these materials for a second year with teachers using the 
conventional text. 

In September 1963, all students were given SCAT 2 A and the pre-tests PLP-1 
and PLP- 2 . Inadvertently, preliminary forms of the pre-tests were given to some 
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schools and final forms were given to others. Consequently only items common to 
the two forms were used. Items from the PLP-1 and PLP-2 were lumped together 
to form a single pre-test score. In TABLE NUMBER 3 the number of students in 
each mode, as well as the corresponding mean scores on SCAT - verbal, SCAT - 
quantitative, and the pre-test, are presented. 



TABLE NUMBER 3 





N 


Mean of 
SCAT V 


Mean of 
SCAT Q 


Mean of 
Pre-test 


MC 


266 


284.8 


301.1 


12.5 


CR 


223 


282.7 


298.8 


10.6 


H 


642 


284.7 


298.8 


11.2 


9F 


368 


284.2 


3OO.8 


11.4 


IA 


102 


275.1 


291.8 


8.7 



For purposes of post-testing, two new tests were constructed: l) PLP-4 

which contained selected items from PLP-2 and PLP-3; and 2) PLP-5 which 
contained additional selected items from PLP-2 and PLP-3 plus analysis items. 
Since there has been some conjecture as to whether or not programed instruction 
is as applicable for teaching higher level skills as it is for teaching compu- 
tational skills, all items from PLP-4 and PLP-5 were coded as to the level 
of intellectual activity which they measured. Two scores were generated for each 
student: l) Lo-cognitive, measuring intellectual activity equal to or less than 

manipulating (computation); and 2) Hi-cognitive, measuring intellectual activity 
greater than manipulating. For a more detailed description of the levels of 
intellectual activity, see APPENDIX D. 

In comparing performance of students using the several modes of presentation, 
it would be fruitful to examine the interaction of mode and student ability, both 
verbal and quantitative. That is to say, are there differences in the relative 
effectiveness of the several modes at different levels of verbal and quantitative 
ability? 

As was stated earlier, the schools were requested to provide students in the 
upper three quartiles on SCAT, 2A. In fact, as can be seen from the frequencies 
in TABLE NUMBER 4 through TABLE NUMBER 7, almost all students lie in the upper 
two quartiles, with the majority of them lying in the top quartile. Unfortunately, 
this does not provide for a sufficient spread in ability to examine the interaction 
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TABLE NUMBER 4 

QUARTILES ON 
SCAT VERBAL 



TABLE NUMBER 5 

QUARTILES ON 
SCAT VERBAL 



MODE 



\ T 


01 


Q2 


03 


Q4 


\ 


T 


01 


02 


03 


04 




266 


169 


72 


18 


7 




266 


169 


72 


18 


7 


MC 


28.8 


30.5 


26.1 


25.4 


22.3 


MC 


6.1 


6.5 


5.3 


5.4 


4.7 




223 


134 


63 


24 


2 




223 


134 


63 


24 


2 


CR 


24.6 


25.9 


24.2 


19.2 


17-5 


CR 


5-2 


5.5 


5.0 


3.8 


4.0 




642 


407 


147 


66 


22 


MODE 


642 


407 


147 


66 


22 


H 


28.0 


31.8 


23.4 


19.2 


16.4 


6.0 


7.0 


4.7 


3.8 


3-0 




368 


241 


97 


25 


5 




368 


241 


97 


25 


5 


9F 


28.4 


30.2 


26.4 


21.6 


19.0 


9F 


5.9 


6.4 


5.1 


3-7 


2.6 




102 


33 


36 


23 


10 




102 


33 


36 


23 


10 


IA 


21.3 


24.1 


21.4 


17.0 


21.5 


IA 


4.2 


4.8 


3-9 


3.6 


4.5 



Cells contain: 

1) Frequency 

2) Mean (Lo) 



TABLE NUMBER 6 

QUARTILES ON 
SCAT QUANTITATIVE 



\ 


T 


01 


02 


03 


Q4 




266 


145 


91 


23 


7 


MC 


28.8 


32.0 


26.5 


19-7 


19.6 




223 


110 


79 


27 


7 


CR 


24.6 


28.0 


22.9 


17.8 


16.4 




642 


325 


199 


91 


27 


H 


28.0 


32.7 


25.6 


20.4 


15.3 




368 


200 


135 


29 


4 


9F 


28.4 


31.4 


25.7 


21.2 


21.5 




102 


31 


32 


30 


9 


IA 


21.3 


26.1 


21.5 


18.2 


14.1 



Cells contain: 

1) Frequency 

2) Mean (Lo) 



Cells contain: 

1) Frequency 

2) Mean (Hi) 



TABLE NUMBER 7 

QUARTILES ON 
SCAT QUANTITATIVE 



\ 


T 


Q1 


02 


03 


Q4 




266 


145 


91 


23 


7 


MC 


6.1 


7.0 


5-4 


4.0 


3.7 




223 


no 


79 


27 


7 


CR 


5.2 


5.8 


4.9 


3-9 


4.0 




642 


325 


199 


91 


27 


H 


6.0 


7.3 


5.2 


4.2 


2.8 




368 


200 


135 


29 


4 


9F 


5.9 


6.9 


5.0 


3-2 


3-2 




102 


31 


32 


30 


9 


IA 


4.2 


5.3 


4.2 


3-3 


3-3 



Cells contain: 

1) Frequency 

2) Mean (Hi) 
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between mode and ability. Looking at the mean Lo-cognitive and Hi-cognitive 
scores by mode and ability level, in TABLE HUMBER 4 through TABLE NUMBER 
indicates that there may in fact be such an interaction. Mode H, for example, 
seems to become relatively less effective going from higher ability to lower 
ability. This seems to hold for both verbal and quantitative. The question 
remains to be answered, by research other than this. 

Since it was not feasible to compare modes by ability level and. since the 
five groups differed somewhat on ability and even more so on initial performance, 
a covariance technique was used to give estimates of what the scores on the Lo- 
cognitive and Hi-cognitive post-test measures of performance would have been had 
the five treatment groups been comparable on SCAT-verbal, SCAT- quantitative, and 
initial level of performance. These estimates, or adjusted means, were tested 
for significance of difference among treatment groups. (See APPENDIX B. ) 

With the Lo-cognitive post-test measure of performance as the criterion 
variable, the analysis indicated that the following differences were significant 
at least at the .05 level: 

MC > CR 
H > CR 
H > IA 
9F > CR 
9F > IA 

While the difference between MC and IA was not significant, it approached 
significance. The differences between the various combinations of MC, H, and 
9F were not significant and neither was the difference between CR and IA 
significant. Thus MC, H, and 9F tend, to cluster together in their effective- 
ness to teach Lo-cognitive skills. Likewise, CR and IA tend, to be equally 
effective, though significantly less effective than the other three modes. 

With the Hi-cognitive post- test measure of performance as the criterion 
variable, only one difference was found, to be significantly different, H > CR. 
These results tend to indicate that all of the modes are equally effective for 
teaching Hi- cognitive skills. 

In TABLE NUMBER 8, the adjusted means for the Lo-cognitive and the Hi- 
cognitive measures are presented. It is evident that even for those differences 
which were significantly different, the differences are small in amount. 
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TABLE NUMBER 8 



MODE 


ADJUSTED MEANS 


Lo- cognitive 


Hi-cognitive 




MC 


27.31 


5.68 


CR 


25.26 


5-35 


H 


28.00 


6.01 


9F 


27.90 


5.72 


IA 


25-70 


5.38 



While the results involving modes MC , CR, and $F do not duplicate the 
results obtained with the 1962-63 data, it must be pointed out that the two years 
of experimentation were not the same. In the 1962-63 experimentation, teachers 
used the MC and CR texts for the first time. In the 1963-64 experimentation, 
only those teachers who had taught the MC and CR texts the previous year were 
allowed to use them for the following year. The failure of the 1963-64 results 
to duplicate those of the preceding year might possibly be due to this difference. 
On the other hand, it might reflect the more sensitive criterion measures and the 
inclusion of verbal and quantitative ability as covariates. 

In February of 1964, and again in May of 1964, all students were administered 
an opinion questionnaire with respect to the text which they were using. One of 
the questions in each of the questionnaires dealt with the frequency with which 
students were required to get help from their teacher. The results of this ques- 
tionare presented in TABLE NUMBER £. It can be seen that there is little dif- 
ference by mode, though the need for help seems to increase later in the year 
for students in all modes. 



TABLE NUMBER 9 



Need Help 
From Teacher 


FEBRUARY 


MAY 




MC 


CR 


H 


9F 


IA 


MC 


CR 


H 


9F 


IA 


Very often 


'-s' 


27 


2k 


3^ 


30 


38 


48 


41 


47 


44 


Seldom 


6 l 


60 


64 


56 


66 


52 


^3 


49 


45 


47 


Not At All 


11 


11 


05 


06 


02 


06 


06 


03 


03 


04 


No Response 


03 


02 


06 


04 


02 


04 


03 


07 


05 


05 



IYGURES ARE IN PERCENT 




Another three questions in each of the questionnaires dealt with confidence 
in using "mathematical vocabulary", "mathematical skills',' and "mathematical con- 
cepts". The results of these three questions are presented in TABLE NUMBER 10 
through TABLE NUMBER 12 . 



TABLE NUMBER 10 



Confidence in 
Using "Mathematical 
Vocabulary" 


FEBRUARY 


MAY 


MC 


CR 


H 


9 F 


IA 


MC 


CR 


H 


9 F 


IA 


Not Confident 


16 


24 


15 


16 


13 


32 


34 


20 


26 


37 


Fairly Confident 


68 


65 


71 


70 


81 


58 


58 


69 


66 


56 


Very Confident 


15 


10 


13 


13 


06 


09 


07 


10 


07 


06 


No Response 


01 


01 


01 


01 


00 


01 


01 


01 


01 


01 



FIGURES ARE IN PERCENT 



Both in February and May, the majority of the students were fairly confident in 
using "mathematical vocabulary". The differences by mode were greatest at the 
end of the first semester, ranging from 65$ of the students in CR indicating 
"fairly confident" to 8 l $ in the IA group. The percent in the "fairly con- 
fident" category diminished, by the end of the year in May, with those in the H 
group maintaining its percent most nearly the same (from 71 $ to 69$). This 
group also lost the least grounds due to "spilling" over into the "not confident" 
category. 



TABLE NUMBER 11 



Confidence in 
Using "Mathematical 
Skills" 


FEBRUARY 


MAY 


MC 


CR 


H 


9 F 


IA 


MC 


CR 


H 


9 F 


IA 


Not Confident 


20 


30 


18 


18 


22 


33 


4 k 


27 


25 


41 


Fairly Confident 


58 


59 


60 


51 


58 


51 


47 


59 


6l 


46 


Very Confident 


21 


10 


22 


29 


19 


15 


08 


13 


13 


12 


No Response 


01 


01 


00 


01 


01 


01 


01 


01 


01 


01 



FIGURES ARE IN PERCENT 
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Again, those in the H group seemed to he the most stable in the "fairly con- 
fident!' category. The two groups which seemed to have suffered most with respect 
to confidence in using "mathematical skills" were those for constructed response 
(CR) and introduction to algebra (IA). By the end of the year, the population 
in the "not confident" and "fairly confident" categories were about the same for 
these groups — approximately b'j'fo in each category. 



TABLE NUMBER 12 



Confidence in 
Using "Mathematical 
Concepts" 


FEBRUARY 


MAY 


MC 


CR 


H 


9F 


IA 


MC 


CR 


H 


9F 


IA 


Not Confident 


17 


28 


16 


18 


22 


25 


39 


25 


28 


35 


Fairly Confident 


59 


60 


65 


57 


65 


60 


50 


63 


60 


56 


Very Confident 


22 


11 


18 


23 


13 


13 


08 


11 


10 


07 


No Response 


02 


01 


01 


01 


00 


02 


03 


01 


02 


01 



FIGURES ARE IN PERCENT 



Again it can be seen that there is little difference by mode, though confidence 
seems to decrease a small amount later in the year. 

In interpreting the preceding four tables it is important to recall that 
there are differences among the modes on ability and pre-test performances. The 
differences are slight for modes MC, CR, H, and 9F and somewhat larger for 
mode IA. See TABLE NUMBER _3. 

The May administration of the questionnaire asked for agreement or disagree- 
ment with the statement, "I would like to have more textbooks like the one we 
used this year". The results are presented in TABLE NUMBER 13. It is quite 
interesting to note that mode H is the only mode for which more students agreed 
than disagreed to the statement. This suggests the possibility that, while sev- 
eral of the modes may be equally effective in presenting the material, mode H 
is the better liked, possibly less boring. 
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TABLE NUMBER 13 



Agree 


”1 would like to have more ma 
books like the one we used th 


,th text- 
lis year". 


MC 


CR 


H 


9F 


IA 


33 


25 


51 


35 


30 


Disagree 


6 1 


70 


43 


59 


63 


No Response 


05 


04 


05 


05 


06 



FIGURES ARE IN PERCENT 



In TABLE NUMBER 14, the mean scores on SCAT- verbal, SCAT- quantitative , 
Lo-cognitive post-test, and. Hi-cognitive post-test, by mode and by response to 
the preceding question, are presented. While there was little difference in 
verbal or quantitative ability between those students who agreed and those who 
disagreed with the statement, there was a larger difference in post-test 
performance . 



TABLE NUMBER 14 





"I would like to have more math text- 
books, like the one we used this year. 


MC 


CR 


H 


9 F 


IA 




SCAT V 


284.75 


283.55 


286.13 


284.80 


277.35 




SCAT Q 


301.33 


300.61 


301.03 


302.61 


294.90 


Agree 


Lo-cog. 


31.49 


27.00 


30.24 


30.61 


23.42 




Hi-cog. 


6.80 


5.77 


6.56 


6.45 


4.74 




SCAT V 


285.17 


282.39^ 


282.53 


283.38 


273.08 




SCAT Q 


301.29 


297.97 


295.95 


299.83 


290.15 


Disagree 


Lo-cog. 


27.64 


23.66 


25.10 


27.00 


20.17 




Hi-cog. 


5.78 


4.95 


5.35 


5.48 


3.91 



One of the questions in the February questionnaire dealt with the general 
impression of the course, i.e., favorable - unfavorable. The results of this 
question are presented in TABIE NUMBER 15. Another measure of attitude toward 
the course, as taken from unstructured COMMENTS, is presented in TABIE NUMBER l 6 . 
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TABLE NUMBER 15 



AIUJJJ. C O £> -LUil Ui 

the Course 


MC 


CR 


H 


9F 


IA 


Very Unfavorable 


05 


06 


02 


02 


00 


Not Favorable 


14 


22 


08 


10 


11 


Neither Favorable 
Nor Unfavorable 


24. 


23 


26 


30 


35 


Favorable 


44 


39 


49 


47 


47 


Very Favorable 


13 


08 


13 


10 


07 


No Response 


00 


01 


01 


01 


00 



FIGURES ARE IN PERCENT 



In each mode except in CR, those whose impressions were either "favorable" or 
very favorable were in the majority. Form H accounts for the highest total 
in these two categories, but it differs only by % from Mc or 9F# 



TABLE NUMBER 1 6 



Attitude 
Toward the 
Course 




FEBRUARY 


MAY 


MC 


CR 


H 


9F 


IA 


MC 


CR 


H 


9F 


IA 


Negative 


13 


19 


11 


08 


14 


27 


35 


17 


17 


32 


Neutral 


21 


17 


23 


22 


10 


35 


28 


45 


42 


15 


Positive 


24 


19 


24 


24 


18 


14 


09 


13 


10 


07 


No Response 


42 


45 


42 


46 


58 


23 


28 


25 


30 


46 



FIGURES ARE IN PERCENT 



Attitudes toward the material appear to be generally favorable. There is little 
difference by mode except that, throughout the year, mode CR is looked upon 
somewhat less favorably and, toward the end of the year, mode IA is also looked 

upon somewhat less favorably. This is in agreement with the results of the main 
analysis. 
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In the May questionnaire, two questions dealt with the general impression 
of the course as regards to interest and difficulty. The results of these 
questions are presented in TABLE NUMBER 17 and TABLE NUMBER l8. 

TABLE NUMBER 17 



Impression of the Course 


MC 


CR 


H 


9F 


IA 


Very Easy Most of the Time 


01 


01 


01 


00 


04 


Easy Most of the Time 


09 


09 


13 


11 


10 


Neither Easy nor Difficult 


38 


31 


32 


32 


29 


Difficult Most of the Time 


37 


38 


32 


38 


33 


Very Difficult Most of the Time 


06 


09 


06 


07 


08 


No Response 


09 


11 


15 


11 


16 



FIGURES ARE IN PERCENT 



TABLE NUMBER l8 



Impression of the Course 


MC 


CR 


H 


9F 


IA 


Very Interesting 


07 


07 


09 


08 


09 


Interesting 


29 


18 


38 


40 


27 


Neither Interesting nor Uninteresting 


24 


29 


24 


24 


24 


Uninteresting 


16 


17 


07 


07 


12 


Very Uninteresting 


10 


06 


05 


06 


07 


No Response 


14 


23 


16 


13 


20 



FIGURES ARE IN PERCENT 



There appears to he little difference among the modes on these two variables. In 
general, the materials tend to be considered somewhat difficult and interesting. 

In TABLE NUMBER 19, SCAT-verbal, SCAT quantitative, Lo-eognitive, Hi-cognitive 
and perceived difficulty are correlated with interest. On the supposition that the 
bright students, the high achievers, and those who perceive the course as easy 
would find the programed materials boring, one would expect to find negative corre- 
lations. The results, however, tend, not to support this supposition. 



TABLE NUMBER 19 





Uninteresting - 


Interesting 




MC 


CR 


H 


9F 


IA 


SCAT-verbal 


.02 


-.13 


.07 


•13 


.12 


SCAT- quantitative 


.15 


.15 


.19 


.10 


.24 


Io-cognitive 


• 25 


.40 


.28 


• 34 


.27 


Hi-cognitive 


.26 


• 37 


.22 


.29 


.12 


Difficult - easy 


•^3 


.21 


.27 


• 37 


• 39 | 



FIGURES ARE PRODUCT MOMENT COR RELA TIONS 



Similarity , ability, performance and perceived difficulty are correlated with 
the general impression of the course (unfavorable - favorable), as measured in 
the May questionnaire, in TABLE NUMBER 20. They are correlated with the 
attitude toward the course (negative - positive), as gotten from unstructured 
COMMENTS, in TABLE NUMBER 21 . 



TABLE NUMBER 20 





Unfavorable - 


Favorable 




MC 


CR 


H 


9F 


IA 


SCAT-verbal 


.10 


.07 


• 13 


-.05 


.08 


SCAT- quantitative 


.07 


.09 


.22 


.09 


.29 


Io-cognitive 


.21 


.17 


.27 


.24 


• 37 


Hi- cognitive 


• 23 


.25 


.22 


.18 


.24 


Difficult - easy 


•35 


.18 


.29 


• 34 


• 31 



FIGURES ARE PRODUCT MOMENT CORRELATIONS 
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TABLE NUMBER 21 



SCAT-verbal 


ATTITUDE TOWARD COURSE: negative - positive 


FEBRUARY 


MAY 


MC 


CR 


H 


9F 


IA 


MC 


CR 


H 


9F 


IA 


.01 


.00 


-.02 


.01 


.CO 


-.07 


.18 


.08 


.07 


.28 


SCAT- quantitative 


-.01 


.04 


.13 


.07 


.23 


-.03 


.10 


.25 


.07 


.04 


Lo-cognitive 


.12 


.04 


.08 


• 13 


.22 


.10 


.17 


.23 


.09 


.08 


Hi- cognitive 


.04 


.05 


.06 


.10 


.20 


.05 


.20 


.18 


.09 


.27 


Difficult - easy 


,1 6 


.19 


.14 


.15 


.40 


• 30 


.22 


.25 


.23 


• 33 



FIGURES ARE PRODUCT MOMENT CORRELATIONS 

The correlations, while generally quite low, tend to "be positive. That is, stu- 
dents who are high in ability, who are high achievers, and who perceive the course 
as easy have a favorable attitude toward the course. 

In order to examine the degree to which performance was dependent on ability, 
the Lo-cognitive and Hi-cognitive measures were correlated with SCAT-verbal and 
SCAT- quantitative. These results are presented in TABLE NUMBER 22. In addition, 
the unstructured COMMENTS were coded for spelling errors and grammar errors. 

These were correlated with the Lo-cognitive and Hi-cognitive measures. The re- 
sults are presented in TABLE NUMBER 23 and TABLE NUMBER 24. Ifode H seems 
to be somewhat more highly correlated, with SCAT-verbal than are the other modes. 
All modes are rather highly correlated with SCAT- quantitative. The correlations 
of the Lo-cognitive and Hi-co^nitive measures with spelling and grammar errors 
tend to be quite low. 

TABLE NUMBER 22 



Lo-cognitive 


SCAT-verbal 


SCAT- quantitative 


MC 


CR 


H 


9F 


IA 


MC 


CR 


H 


9F 


IA 


• 34 


.29 


•59 


.41 


• 32 


•57 


.54 


.6o 


•53 


.62 


Hi-cognitive 


•30 


• 23 


•52 


.42 


.19 


.49 


.40 


•54 


.49 


.44 



FIGURES ARE PRODUCT MOMENT CORRELATIONS 
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TABLE NUMBER 23 



Lo- cognitive 


SPELLING ERRORS IN COMMENTS 




F] 


EERUARY 


MAY 


MC 


CR 


H 


9F 


IA 


MC 


CR 


H 


9F 


IA 




.08 


.14 


.18 


.01 


.09 


.13 


-13 


.28 


.19 


Hi- cognitive 


.03 


.Q5 


.14 


.10 


■03 


.07 


.17 


.07 


.22 


-.05 



FIGURES ARE PRODUCT MOMENT CORRELATIONS 
TABLE NUMBER 24 




Lo- cognitive 



Hi- cognitive 



FIGURES ARE PRODUCT MOMENT CORRELATIONS 



In addition to performance, it is of interest to consider the relative cover- 
age of the five modes. In TABLE NUMBER £ the percent of classes having finished 
the material through Chapters 14, 15 , and. 16-17 as of June 3 . 5 th is presented. 
While no mode accomplished, anything near complete coverage of the text, mode H 
appears to offer the best coverage, with mode MC second best. These are results 
similar to those found with regard to performance. 



TABLE NUMBER 25 





THRU 14 


THRU 15 


THRU 16-17 


MC 


47 


26 


21 


CR 


62 


15 


8 


H 


55 


42 


30 


9F 


23 


9 


5 


IA 


29 


0 


0 



FIGURES ARE IN PERCENT 
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Follow-up Study of Retention (May 1964) 



In May of 1964, about 450 students who had been in the 1962-63 classes 
were administered a retention test (PIP-4) to determine whether the algebra 
they had studied was related to the mode they had used. These students were 
regular 9'th graders studying on a time basis with a teacher. The students 
selected were randomly chosen from classes which had used modes MC, CR, or 
9F in 1962-63. 

Each student was asked to indicate the type of mathematics he was studying 
the following year. In TABLE NUMBER 26 the number of students, as well as 
their mean score on PLP-4, is presented for each category of mathematics being 
studied within each mode. 



TABLE NUMBER 26 



Type of Math Taken 
the Following Year 


MODE 


MC 


CR 


9F 




147 


106 


39 


Geometry 


15.5 


17.1 


15.3 




27 


13 




Algebra 


12.9 


16.7 


— 




6 


16 




Other 


T±3 . 


7.6 


-- 




15 


38 


12 


No Math 


8.6 


9.7 


9.6 




33 


5 


1 


No Response 


21.7 


13.3 


4.0 




220 


176 


52 


Total 


15.4 


14.6 


13.8 



Cells Contain: 

1) Frequency 

2) Mean (PLP-4) 



For testing the significance of difference among the three groups, a co- 
variance technique was again used. PLP-4 was the criterion variable and. PLP-1 
and PLP-2, given in September of 1962, were the covariates. The null hypothesis 
of no difference among treatments after adjusting with covariates was not disproved. 
(See APPENDIX C.) That is to say, that one year after completing the course, per- 
formance on PLP-4 was not significantly different for students who had used dif- 
ferent modes, i.e., MU, CR or 9F. In TABLE NUMBER 27 the means of PLP-4, 
adjusted for PLP-1 and PIP-2, are presented. 
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TABLE NUMBER 27 



MODE 


ADJ. MEAN 
PLP-lj- 


MC 


1 L -.90 


CR 


15.06 


9F 


ih.ki 



While it might have proved interesting to look at retention in terms of 
Lo-cognitive and Hi-cognitive performance, PLP-4- contained an insufficiently 
large enough number of Hi-cognitive items to generate such a score. 

Final Revision of the Hybrid (Summer 196 ^) 

During the 196 ^ SMSG Summer Writing Session, a writing team chaired by 
D. Blakeslee and consisting of W. G. Chinn, F. Jacobson, M. Matchett, W. Matson, 
and. P. Redgrave comprehensively revised the hybrid text. This revision included 
significant changes in the mathematical content of 9F based on critical reviews 
by the PEP writers and. analytical comments from members of the SMSG Advisory 
Board, notably those of P. D. Lax, New York University, and H. 0. Poliak, Bell 
Telephone Laboratories. In addition, a Teacher Commentary was prepared. 

During the 1963 - 6 ^ testing, as each chapter was completed, teachers adminis- 
tered the SMSG-PLP chapter tests and returned the answer sheets for analysis. In 
support of the revision, a separate random sample of 100 of these answer sheets 
were pulled for each chapter and mode, and a comprehensive item analysis was pre- 
pared. Furthermore, for about 100 students using m*de H and about 100 stu- 
dents using mode 9F, a record, was kept of the number of errors per item and of 
the total scores on PLP-1, 2, k, and. 5 and on each of the chapter tests. 
Finally, teacher reaction questionnaires for each chapter and mode were utilized. 

The final revision of the hybrid text concluded SMSG’s examination of pro- 
gramed instruction. The original literature on programing does not provide for 
a program "mix". However, the mixed form tried in the 196 ^ writing has more 
clearly met the criteria of improved student performance and greater philosophical 
satisfaction to the writers. In view of this, an examination of programed mater- 
ials that are feasible and promising should include such hybridization for 
consideration. 
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APPENDIX A 
Test Score Estimates 

Since the treatment groups are not comparable on the pre-tests, analysis 
of covariance is appropriate for adjusting the post-tests for the effect of the 
differences on the pre-tests and for testing the significance of over-all dif- 
ferences among the treatment groups on the post-tests. 

Case I pip_ 2 (post) is the dependent variable and PLP-1 (pre) and PIP-2 (pre) 
are the covariates. 



The adjusted means 


are: 


CR 


16.6580 


MC 


16.2195 


MCR 


13.8592 


9F 


14.8990 


The variance ratio 


is: 



F(3, 875) = 279-4771 / 18.9009 = 14.79 
CRITICAL VALUE (.001 level) = 5*42 

CASE II PLP- 3 (post) is the dependent variable and PLP-1 (pre) and PLP-2 (pre) 
are the covariates. 



The adjusted means 


are: 


CR 


10.7214 


MC 


10.9281 


MCR 


9.2799 


9F 


9.8927 


The variance ratio 


is: 



H3 , 875) = 102.1206 / 12.2316 = 8.35 

CRITICAL VALUE (.001 level) =5.42 




The analysis of covariance model assumes homogeneity of variance on the depen- 
dent variable. Applying Bartlett’s test, we find: 

Case I For PIP- 2 (post) 

v= 5.50 

CRITICAL VAHJE (.01 level) = 11. 34 
Case II For PLP- 3 (post): 

v = 52.83 

CRITICAL VALUE (.01 level) = 11. 34 

Since the assumption of homogeneity of variance does not hold under Case II, 
caution should be observed in interpreting the analysis, particularly since 
classes were formed without random assignment. The standard deviations of 
PLP-3 (post) for the four treatment groups are as follows: 



CR 


3.82 


MC 


5.06 


MGR 


3.50 


9F 


3.30 



Since F, as an over-all test, indicates significant differences among the 
groups, we may safely make further tests to see whether selected pairs of means 
differ significantly. This involves the computation of D / S D for each pair, 
where D is the difference between the adjusted means and S is the standard 
error of the difference between the adjusted means. 

Since we have no a priori hypotheses for making comparisons between selected 
pairs of means, the t test is inappropriate for testing D / S in that we 
would tend to capitalize on change- large differences and consequently lower our 
chosen level of significance. Instead, we will use Scheffe’s S-method which 

requires that D / be equal to or greater than K if we are to consider the 
difference significant, where K is defined as: 

K= /Tg-D F a 

with a being the chosen level of significance and df : n = G-l and n = N-G-2. 
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For a 



df: 



Case I 



.001, K = 


•t (3)(5^2) 


= 1*.03 


.01, 


V (3)0-78) 


= 3-3Y 


.05, 


V(3)(2.60) 


= 2.79 


3 and n g 


= 875- 





Case II 



PLP-2 (post) is the dependent variable and PIP-1 (pre) and PLP-2 (pre) 
are the covar?ates. 



1 ) 


CR - 9F: 


D / Sp = 1.7590 / 0.1*628 = 3-80 


2 ) 


MC - 9F: 


1.3205 / 0.1*511 = 2.93 


3) 


9F - MCR: 


1.0398 / 0.5199 = 2.00 


*0 


CR - MC: 


0 . 1*385 / 0.3602 = 1.22 


5) 


CR - MCR: 


2.7988 / 0.1*605 = 6.08 


6 ) 


MC - MCR: 


2.3603 / 0.1*1*92 = 5.25 


PIP- 3 (post) is 


the dependent variable and PIP- 3 


are 


the covariates. 


i) 


CR - 9F: 


D / S D = 0.8287 / 0.3723 = 2.23 


2 ) 


MC - 9F: 


I.O 35 I* / 0.3629 = 2.85 


3) 


9F - MCR: 


0.6128 / 0 . 1*183 = 1 . 1*6 


4) 


MC - CR: 


0.2067 / 0.2897 = 0.71 


5) 


CR - MCR: 


1.1*1*15 / 0.3701* = 3.89 


6 ) 


MC - MCR: 


1 . 61*82 / O .3613 = ^.56 
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APPENDIX B 



Lo-Hi Cognitive Estimates 

Since the treatment groups are not comparable on student ability (as meas- 
ured by SCAT, 2A) or on the pre-test, analysis of covariance is again appropriate 
for adjusting the post-test measures for the effect of the differences in ability 
and pre-test performance and for testing the significance of over-all differences 
among the treatment groups on the post-test measures. 

Case I The Lo-cognitive post-test measure is the dependent variable and SCAT- 
verbal, SCAT- quantitative, and pre-test performance are the covariates. 



The adjusted means 


are: 


MC 


27.3148 


CR 


25.2633 


H 


28.0039 


9F 


27.9023 


IA 


25.6955 



The variance ratio is: 

F(4, 1593) = 402.2607 / 38.2326 = 10.52 
CRITICAL VALUE (.001 level) =4.62 

Case II The Hi- cognitive post- test measure is the dependent variable and SCAT- 
verbal, SCAT- quantitative, and pre-test performance are the covariates. 



The adjusted means 


are: 


MC 


5.6823 


CR 


5-3464 


H 


6.0121 


9F 


5.7188 


IA 


5.3821 



The variance ratio is: 

F(4, 1593) = 23.6080 / 4.264 = 5.50 
CRITICAL VALUE (.001 level) =4.62 
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The analysis of covariance model assumes homogeneity of variance on the dependent 
variable. Applying Bartlett’s test, we find: 









Case I For Lo-cognitive post- test measure: 

v= 5.69 

CRITICAL VALUE (.01 level) = 13.28 



Case II For Hi-cognitive post- test measure: 

V = 4.21 

CRITICAL VALUE (.01 level) = 13.28 



Thus we find the assumption of homogeneity of variance satisfied for both cases. 

Since F, as an over-all test, indicates significant differences among the 
groups, we may safely make further tests to see whether selected pairs of means 
differ significantly. We could, on the basis of the previous year’s results and 
on theoretical consideration, make specific hypotheses regarding selected pairs 
of means and test these hypotheses via t tests. The 1962-63 experimentation, 
however, is sufficiently different from the 1963-64 experimentation that it seems 
more desirable to make all possible comparison and. to treat those parts of the 
two years’ experimentation that are alike as a replication. Testing all poss- 
ible comparisons makes the t test inappropriate in that we would, tend to 
capitalize on chance-large differences and consequently lower our chosen level 
of significance. Again we will use Scheffe’s S-method which requires the D / 

(the difference between the adjusted means divided by the standard error of the 
difference) be equal to or greater than K if we are to consider the difference 
significant, where K is defined as: 



K = V(G-l) F a 

with a being the chosen level of significance and 



For a = .001, K = 

= . 01 , 

= .05, 

df : n^ = 4 and n^ 



V(*0(t.62) 

V00(3.32) 

V(4)(2.37) 

1,595. 



df: 



»• 






= 4.30 
= 3.65 
= 3.08 



n = G-l 



and 



n 2 = N-G-2. 









■*• .^*r*vy ~*? r ~‘J&'>£' r^aw?S 



Case I 



The Lo-cognitive post-test measure is the dependent variable and SCAT- 
verbal, SCAT- quantitative, and pre-test performance are the covariates, 



Case II 



1 ) 


H - MC: 


D / S D = 0.6891 


/ 0.4542 = 1.5171 


2 ) 


H - CR: 


2 . 7^06 


/ 0.4817 = 5.6894 


3) 


H - 9F: 


0.1016 


/ 0.4056 = 0.2504 


4) 


H - IA: 


2.3084 


/ 0.6714 = 3.4381 


5) 


MC - CR: 


2.0515 


/ 0.5659 = 3.6251 


6 ) 


9F - MC: 


0.5875 


/ 0.4999 = 1.1752 


7) 


MC - IA: 


1.6193 


/ 0.7373 = 2.1962 


8 ) 


9F - CR: 


2.6390 


/ 0.5257 = 5.0199 


9) 


IA - CR: 


0.4322 


/ 0.7468 = 0.5787 


10 ) 


9F - IA: 


2.2068 


/ 0.7050 = 3-1302 


The 


Ki- cognitive post-test measure is the dependent variable and SCAT- 


verbal, SCAT- quantitative, and 


pre-test performance are the covariates. 


1 ) 


H - MC: 


D / S D = 0.3298 


/ 0.1522 = 2.1668 


2 ) 


H - CR: 


O .6657 


/ 0.1615 = 4.1219 


3) 


H - 9F: 


0.2933 


/ O.I 36 O = 2.1566 


4) 


H - IA: 


O. 63 OO 


/ 0.2251 = 2.7987 


5) 


MC - CR: 


0.3359 


/ 0.1897 = 1.7706 


6 ) 


9F - MC: 


O. 365 O 


/ O.I 676 = 2.1778 


7) 


MC - IA: 


0.3002 


/ 0.2472 = 1.2144 


8 ) 


9F - CR: 


0.3724 


/ 0.1762 = 2.1135 


9) 


IA - CR: 


0.0357 


/ O. 25 O 3 = 0.1426 


10 ) 


9F - IA: 


0.3367 


/ O .2363 0 1.4248 



37 



r j smx&VBi****** 



O 



f 



APPENDIX C 



Estimates of Retention 



Since the treatment groups are not comparable on the pre-tests (given in 
September of 1962), analysis of covariance is appropriate for adjusting the 
post-test (given in May of 1964) for the effect of the differences on the pre- 
tests and for testing the significance of over-all differences among the treat- 
ment groups on the post-test. 

PEP-4 is the dependent variable and. PEP-1 and. PEP- 2 are the covariates. 



The adjusted means 


are: 


MC 


14.90 


CR 


15.06 


9F 


14. 4l 



The variance ratio is: 

F(2, 451) = 8.6077 / 21.2961 = 0.40 
CRITICAL VALUE (.001 level) =6.91 
CRITICAL VALUE (.05 level) =2.99 

The analysis of covariance model assumes homogeneity of variance on the dependent 
variable. Applying Bartlett* s test, we find: 

V = 8.4l 

CRITICAL VALUE (.01 level) = 9.21 
Thus we find the assumption of homogeneity of variance satisfied. 
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APPENDIX D 



Levels of Intellectual Activity 



* 

k 

i 

} 

i 

i 

i 



1) 

2 ) 



KNOWING - Knowing terminology, facts, and rules 



TRANSLATING - 



1. 

2 . 

3 - 



Changing from one language 
Expressing ideas in verbal 
Codifying patterms 



to another 

f symbolic, or geometric form 



• 2 ) MANIPULATING (Computation) - 

1 . Carrying out algorithms 

2. Using techniques 



4) CHOOSING - 



1. Making comparisons . 

2. Selecting appropriate facts and. techniques 

3 . Guessing 
k. Estimating 

5 . Changing one’s approach 

6 . Selecting new symbolism 



5 ) ANALYZING - 



1. Analyzing data 

2. Finding differences 

3. Recognizing relevant and irrelevant data . 
k. Seeing patterns, isomorphisms, and symmetries 

5 . Analyzing proofs 

6 . Recognizing need for additional information 
7*. Recognizing need for proof or counterexample 



6 ) 



SYNTHESIZING - 1. 

2 . 

3 . 

h. 



Specializing and generalizing 
Conjecturing 
Formulating problems 
Constructing a proof or a problem 



7 ) EVALUATING - 



1. Validating answers and the solution process 

2. Judging reasonableness of answers 

3 . Criticizing proofs 

4. Judging the significance of a problem 
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APPENDIX E 



Sample Summary Guide For Writing 

Chapter 12 - Polynomial and Rational Expressions 

In this chapter , the concept of a polynomial is introduced and it is seen 
that the problem of factoring polynomials is most interesting when restricted to 
factors which are themselves polynomials. Most of the work is with polynomials 
over integers i.e., polynomials whose coefficients are integers. Polynomials 
over the rational and real numbers are also considered. Each of these sets of 
polynomials is closed under addition and multiplication. 

The problem of factoring is to write a given polynomial, which is considered 
to be of a certain type, as an indicated product of polynomials of the same type. 

A polynomial which is not factorable as a polynomial over the integers may or may 
not be factorable when regarded as a polynomial over the real numbers. Factoring 
a polynomial over the rational numbers can be reduced to factoring a polynomial 

over the integers. 

It was found that factoring is a useful tool for solving equations. The 
various methods of factoring are all just applications of the distributive pro- 
perty. Besides immediate change of an indicated sum into an indicated product, 
we have the difference of squares, perfect squares, completing the square, and 
quadratic polynomials in one variable. The technique for factoring quadratics 
is based strongly on the knowledge which has been obtained about factors of num- 
"hers, and minimizes guess work# 

The concept of a rational expression is then considered and it is observed 
that rational expressions have the same relationship to polynomials as rational 
numbers have to integers. Problems of simplifying rational expressions are 
similar to the problems for rational numbers. It is seen that the rational ex- 
pressions have the usual properties of fractions and that factoring of polynomials 
plays the same role in the work with rational expressions as factoring of integers 
plays in the work with rational numbers. 

Every rational expression can be written as an indicated quotient of two 
polynomials which do not have common factors. 
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A systematic method for division of polynomials in one variable is devel 
oped. This is based on the following important property of polynomials: For 

any two polynomials N and D with D different from zero, there exist poly- 
nomials Q and R, with R of lower degree than D, such that N = QD + R. 
The process of division gives us a way of calculating 
D are given. 



Q and R when N and 
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PREFACE 



In 1961, the School Mathematics Study Group decided to undertake a project in 
programed learning with special reference to the kind of mathematics illustrated 
in the texts prepared by the SMSG. The Project has been under the supervision of 
a Panel consisting of: R. C. Buck, University of Wisconsin; E. E. Hammond, Jr., 

Phillips Academy (Andover); L. D. Hawkinson, San Francisco Public Schools; 

J. G. Holland, Harvard University; W. J. McKeachie, University of Michigan; 

H. N. McNeille, Case Institute of Technology; H. 0. Poliak, Bell Telephone 
Laboratories; and D. W. Taylor, Yale University. The Project has been coordi- 
nated by Leander W. Smith, Assistant to the Director of the SMSG, E. G. Begle. 

The Panel decided as a first step to have the mathematics of the SMSG F irst 
Course in Algebra translated into programed form. Two program formats were 
chosen by the Panel in 1961: (l) The first of these consisted of a program 
based on the constructed response linear programs developed by B. F. Skinner, 
Harvard psychologist; (2) The second of these consisted of a program based on 
the scrambled book, multiple- choice (intrinsic) programs developed by N. Crowder 
of U. S. Industries. In the spring of 1963 a hybrid format was proposed by the 
coordinator which attempted to combine the features of both of the earlier pro- 
grams and retain the content of the First Course in Algebra . The latter program 
was revised in the summer of 1964 for release to schools in the fall of 1964. 

In preparing the programs, the usual SMSG procedures were followed. The 
programing was done by mathematicians, teachers of mathematics, and psychologists 
working together on writing teams. The materials were tested in a variety of 
classroom situations and revised on the basis of the experience gained. Such 
materials as prove effective will be made generally available; however, perhaps 
more important will be any explicit and detailed information on programing pro- 
cedures, classroom experiences, techniques for using programs which will be made 
available from time to time as the project progresses. 

During the summer of 1961, a Manual was prepared by the project coordinator 
to be used as a basic document in this effort. It was designed to supply the 
individuals who were already in command of the mathematics with the information 
that they would need for the actual construction of programed texts. The pre- 
sent Manual contains much that was in the earlier version and, in addition, draws 
heavily on examples of programing done in the 1961-64 years of the Project. As 
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programed learning evolves from its earlier stages, so too this Manual -will need 
to reflect and anticipate further changes. Suggestions for the improvement of 
this Manual should he sent to: SMSG Programed Learning Project, SMSG - Stanford 

University, Stanford, California. 
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1-0 Introduction 



O 

ERIC 



This Manual is written for prograniers (and potential programers) whose spe- 
cific assignment is the preparation of programed texts in mathematics based on 
the work of the SMSG. Many problems are presented and topics discussed which 
have failed to appear in other sources; their appearances here will be justified 
if they shed light on the psychology behind and the methodology of programed 
instruction. 

1-1 Historical Background 

The storm and wild enthusiasm experienced in the past few years over "auto- 
mated teaching” has some of its roots in the classical tradition. Prom the 
Platonic dialogue came the Socratic approach of directing students to desired 
goals by asking questions; from the colonial hornbook came the incremental step 
approach; and from many sources came the practice of rewarding students who 
mastered their lessons. But programed instruction transcends simple reward 
techniques and increments of rote learning; programed instruction is proposed 
as an application in the classroom of a theory developed, in psychological lab- 
oratories (see J. G. Holland: “Teaching Machines : An application of Principles 
from the laboratory” in Lumsdaine and Glaser: Teaching Machines a^d Programmed 
l earning : A Source Book , pp. 215-29) and. self-instruction and trouble-shooting 

techniques in military training bases. 

In 1926, Sidney Pressey (see Lumsdaine and Glaser, op. eit., pp. 32-51 and 
pp. 69-88) developed a self- scoring device to test students without consuming 
the time and energy of an instructor. He anticipated the device would be useful 
as a teaching instrument; but in 1926 the idea did not take hold and little was 
done to develop the self-instructional devices for almost thirty years. 

With the publication of B. Frederick Skinner* s articles on “The Science of 
Learning and the Art of Teaching” (see Lumsdaine and Glaser, op. eit., pp. 99-H3) 
in 1954 and "Teaching Machines” (see Lumsdaine and Glaser, op. eit., pp. 137-72) 
m 1958, there followed, what many have called an "industrial revolution in 
education". Skinner, a Harvard psychologist, presented the notion of operant 
conditioning as the psychological basis for a type of programed learning. Al- 
though many efforts reproduce the Skinnerian programed format, they tend to 
neglect the essence of his psychology— the shaping of behavior operant (rather 
than reflex) conditioning. To ensure activity and participation of the learner, 
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the Skinner-type program requires the student to construct his answers. The 
Analysis of Behavior , a text in psychology by B. F. Skinner and J. G. Holland, 
provides an illustration of the Skinner-type format which has been widely used. 

While Skinner and his followers continued to investigate constructed re- 
sponses, frequency of response, confirmation mode, overt vs covert responses, 
error rates, and time variables, the multiple- choice mode evolved. Norman 
Crowder (see Lumsdaine and Glaser, op. cit., pp. 286-98) added the corrective 
element and developed "intrinsic programing" which presents a topic to a student 
with a multiple-choice question for which he selects one alternative. If he 
has chosen a correct response, he continues along a "main line" program; if he 
has chosen an incorrect or less correct response, he may receive further exposi- 
tion, enrichment, or be returned to the topic before proceeding along the main 
line. Electro-mechanical machines are capable of handling this type of progr am , 
although a scrambled text may be as effective. 

Teaching machines, as such, shall not be discussed in detail here since 
their role has not been clearly defined at this time. It is not the intent nor 
purpose of this manual to criticize the manufacturers of specific machines or 
programs, but simply to call attention to their work as it has acted as a stimu- 
lant to the SMSG Project. For a partial listing of available machines and 
programs see Rigney and Fry: Audio - Visual Communications Review Supplement 

Three (May-June 1961) or Finn and Perrin: Teacning Machines and Programed 

Learning (1962 TDP of USOE). 

Simple machines include: sliding masks, paper- fed machines, roll type 

machines, punchboards, and variations of these. More complex mechanisms in- 
clude: electric typewriters, audo-visual units, and computer-based machines. 

The role of the computer is not yet fully known. Electronics corporations are 
currently engaged in research to investigate educational programing. What is 
likely to happen in the next decade is open to speculation. At this state, it 
is obvious that speed of operation and the variety of display potential are 
but two of the contributions which a computer-based system can offer. 

While machines and hardware occupy the labors of some, many publishers are 
engaged in the production of texts in "programed, form". Many, if not most, 
appear to be sterile reproductions of early programing efforts. Others are re- 
flecting the widespread programing studies which have sprung up in colleges and 
universities throughout the country with subject matter ranging from statistics 
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to foreign language training. Extensive programing of mathematics by persons 
expert in programing but not in mathematics has led SMSG to begin its study 
with persons thoroughly acquainted, with the mathematics in question and. to 
teach these people to program* 

The call to research has been voiced from many critics and historians of 
the programed learning movemen+. Among them, Wilbur Schramm (in Programed 
Instruction Today and Tomorrow , pp. 73-4) comments: 

"...When we understand more than we do now about how to combine 
different kinds of programing, how to vary the schedule of rein- 
forcement, and how to fit a program to different learning ob- 
jectives and student abilities, above all when we learn more about 
how to maintain the student’s interest and challenge him through 
what is now very often a very dull exercise in conditioning, then 

programs are likely to look far different from the way they look 
today. " 







2-0 The Job Ahead 



Programing is not an easy task! It requires a thorough knowledge of the 
subject, a sensitivity to the ability of the student for whom the material is 
being prepared, and an extraordinary patience. The pearly prose written on 
Monday may be torn apart on Tuesday by a student* s "I don't understand.” The 
time spent writing enough to keep a student busy for one class hour may range 
to five or six hours, not including editorial and revision time spent. Is pro- 
graming an art, or is there a semi- systematic procedure by which the writing 
can be completed? 

Let us examine briefly the procedures used by SMSG in programing the First 
Course in Algebra . 

(1) Content analysis - chapter summaries were prepared by members of the 
original writing team to give programers an idea of what is covered, why it 
occurs at a given time, and what review of material seems required. 

( 2 ) Statement of Objectives - general objectives and sample questions to 
measure the completion of these objectives were prepared (see APPENDIX 4). In 
addition, each programer was required to specify the objectives of the section 
being programed and to list these at the beginning of the draft version. These 
section- by- section objectives could then be checked in the reviewing process. 

( 3 ) Preparation of a first draft by a programer - the first draft of a 
given section was required to preserve the content and sequence of the original 
text. Often an outline of the section was made prior to the writing of material. 
This stage may appear on cards although ditto was more effective and satisfactory. 

(4) Review and criticism by other programers - all members of the team re- 
viewed all of the first several sections to gain uniformity of writing and style 
and see variety of techniques of presenting ideas. 

( 5 ) The "Hearing” - a session at which individual programers defend their 
writing to the group and offer suggested revisions - semantic, schematic, con- 
tent, etc. 

( 6 ) A revised draft is prepared in the style desired. 

(7) A student is found, to go through the section to offer criticism and 
suggestions. 

( 8 ) The team captain reviews the draft, offers minor changes, and marks 
one copy for final typing. 
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(9) The final copy is prepared for photolithography and the text is printed. 

(10) The text is used in a variety of classroom situations with reports to 
the SMSG on each chapter. 

(11) Revision of the text takes place based, in part, on teacher reactions 
and student performances on SMSG tests of content. 

Prom the experience gained in the summer of 1961, 1962, and 1963, we can 
abstract a model of how programing might take place: 

(1) content analysis 

(2) specification of goals 

(3) idealization of the learning sequence 

(4) construction of the first draft (usually by a mathematician) 

(5) revisions and suggestions (usually by teachers) 

(6) incorporation of revisions into second draft 

(7) student tryout 

(8) editing 

(9) preparation of final copy 

But these steps do not seem to differ considerably from the way that a conven- 
tional text is written. The programing of the first draft, however, must be a 
point of difierence at least. In the sections which follow, let us examine the 
steps (1-9 above) in greater detail with some questions to guide authors in 
writing future programs in mathematics. 

2-1 Content Analysis 

"Clearly the programer knows what he‘s doing." This may be, but more often 
than not, he fails to see it in writing until after the task is done. This first 
step in programing is not particularly easy since it involves stating precisely 
what the material to be covered, really is. What terms, concepts, and skills are 
to be mastered? What, if any, order is explicit in the development of the con- 
cepts and/or skills? What order is implicitly agreed upon by other subject area 
specialists as necessary to coverage of the topics? Will an outline of the con- 
tent reveal a subject-oriented order of the content? What topics are assumed, as 
backgrounds to the new topic being developed? What subdivision of the content 
is necessary? 
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2-2 Specification of Goals 

Once the content is clearly defined there is need for explicit statement of 
goals. What should the student he able to do when he has studied this unit? 

What terms should he have mastered? Does mastery mean ’’ability to recognize,” 
’’ability to spell, write, or use,” or ’’ability to reconstruct (as in a proof)”? 
What is the level of the student at the beginning of the program? What tools, 
terms, concepts does he have to work with when he starts? Which of these shall 
he need to have refreshed? What item can we present which will test mastery of 
that which we want to teach? 

Content goals must be explicit. For example, we may wish a student to 
recognize a proof of a theorem, or to perform certain manipulations, or to re- 
produce from memory a statement generalization, or to factor a given expression 
over the real numbers. We may want him to generalize from cases or find a 
specific instance of a given generalization. These all involve the student in 
an activity toward some specific behavioral goal. It would be of little value 
to provide only general objectives in terms such as "know how to solve equations,” 
or ’’understand the number system,” or ’’appreciate the elegance of the proof." 

The teachers’ commentaries for SMSG texts provide sanple test questions to 
illustrate the specific skills, concepts, and vocabulary desired as outcomes of 
studying SMSG mathematics. Although these are useful as a guide, they represent 
only a sampling of the myriad of abilities which a student acquires during an 
academic year. This statement may seem too obvious but programers will do well 
to bear in mind that a program may serve in lieu of the combination of text and 
teacher in the conventional classroom. With this in mind let us move to phase 
three of the programing process. 




2-3 Idealization of the Learning Sequence 

Given a topic, a well-defined student, and time, we now come to the stage 
in programing where teaching experience weighs heavily. A graduate student may 
have a thorough mastery of elementary material and may (and often does) fail to 
communicate to his students for any number of reasons. Among the experienced 
teachers there tends to be a general agreement on the manner of presenting new 
material to students — agreement in the sense that they can spot (without a stu- 
dent present) the areas that will cause trouble and/or prevent learning. 



52 






\ 



/ 





When should a given topic he introduced? What can he done to motivate the 
learning? Would an illustration help? What prior knowledge can he assumed? 

What earlier skills need reinstatement before this topic can he developed? What 
would a master teacher do to insure learning of the topic? 

Idealizing the teaching sequence might go through the following steps: 

(1) Review prerequisite concepts through discussion and/or warm up exer- 
cises. This may involve presenting a flow chart of earlier ideas and definitions 
leading to the new topic. 

(2) Ask a criterion question to determine readiness for the new topic. 

( 3 ) Present ioems which force discrimination between previous learning and 
the new topic. 

(4) Cite examples or instances of the new topic. 

(5/ Utilize diagrams, visual aids, and any available media to refine the 
new concept. 

(6) Determine the level of generalization which the student can and should 
reach. Do not attempt to take all students to equal depth on a topic. A branch 
in the program may he developed to aid. the less able or more able student j a 
reference to material outside the text may serve the same purpose . 

(7) Work through some illustrations. Call for student activity from simple 
to complex cases. 

(8) Test against the original criterion. 

(9) Reteach or return the student to material he failed, to grasp. 

looking at this from a student’s viewpoint may take a similar form: 

(1) What earlier material must I recall? 

(2) How does the new concept fit into my knowledge? 

(3) Am ± ready to learn the new topic? 

(4) Let’s try a few examples. 

(5) What counterexamples can I find? 

(6) Can I find some original applications? 

(7) What test can I use to be sure I know? 

(8) What should, I go back over? 
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The basic element to be considered in this planning stage is — the student 
can learn only if there is material to learn and he will learn only as well as 
the material is presented. The more careful the planning, the better the first 
draft is likely to be. 

On motivation, let a few words be said. The early programing efforts tended 
to believe that if a stud.ent was correct most of the time (95 percent was the 
golden number), he would continue to want to learn. This may be, but low error 
rate cannot be taken alone as an indicator of high learning and motivation. 
Students in the constructed response format text complained of boredom as much 
as those in the multiple -choice format text. Hence in 1963, a variety of pro- 
graming was maintained and the complaints diminished. 

2-4 Writing a First Draft 

Up to this point we have been concerned with the planning stages. In this 
section we shall find it necessary to refer to "frames" and "panels". These 
terms are peculiar to programed texts and will be left almost undefined for the 
moment to see if, in fact, we can build a flexible attitude and an operational 
definition as an outcome of our work. 

Asking a question, we may require a student to construct his response. 

For example, we may ask 



Uhat is the common name for (-3) + 4? 
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The question is enclosed by a frame and the confirmation of the student’s re- 
sponse is given immediately to the right. Let us examine other frames and. 
notice the manner of posing questions and. the location of confirmations: 



Look at the following sentences: 

5 ( 0 ) = 0 
3 ( 0 ) = 0 
0 ( 0 ) = 0 
rr(0) = 0 
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Each time that we find the product of a number and zero 
our result is zero. Consider the sentence, k(0) = 0. 
Is there a number k which will make this sentence 
false? 

The sentence k(0) = 0 is true for every k. 

This property of numbers is called the 
of Q. 

Simply stated: Any number times equals 
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The frame above follows a parallel treatment of the multiplication property 
of 1 and precedes the use of the commutative property to show that: For every 
number a, a(0) = 0 or (0) a = 0. 

The following frames show how a student may be led through a sequence of 
items to tie together some of the notions he has studied. 



Recall that 



reduced to 



2k 



In fact, ~ reduced to lowest terms is 
3& 



terms is — . 



ak 



This is an instance of the general statement that — 

JD bk 

□ , for numbers a, b, k, with b ^ 0 and 



k ^ 0. 



In arithmetic, we noted that 

2 5 _ □ X □ 

37 3X7 * 



In general, ^ X § ■= 



Note that | X £ = 
b k 



ac 



□ 




, and, just as is 



another name for 1, r- is another name for 



We seen then, that ^x^ = ^xl = 

b k b 
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Together, the definition of multiplication of rational numbers, the fact 

cll£ SI 

that ^ = — , and the multiplication property of 1, give us an effective 
procedure for dealing with complex fractions in arithmetic and algebra. For 
example, we may use these to find a common name for: 



! + 5 



4 

7 



Using the multiplication 


2 2 

3+5 f+5 


21 


property of 1: 


4 ~ 4 

7 7 


21 


21 

(We chose because 21 

of and . ) 


is the least common multiple 


We can write: 


! + 5 (f + 5 ) 


(21) 




□ 

1 

it— 


(21) 


Using the distributive 
property: 


= f (□) +5 (□) 

Y (21) 




Multiplying: 


= 14 + □ 
12 




and adding: 
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Some frames require the student to select from alternatives which follow 
a multiple-choice question, as in the following cases: 
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Example 




Write the phrase (5 + a) (a + 8 ) as an indicated sum: 

[A] 40 + a“ 

[B] 5 a + a 2 + 40 + 8 a 

[C] a 2 + 13 a + 40 
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Example : 



If you know that the product of two numbers is 0, 
and that one of the numbers is 3 , the other number 

41 is . 

If the product of two numbers is 0, at least one 

42 of the numbers is 
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In Items 4l and 42, does the multiplication property of 0 
provide complete answers to the questions? 

[A] yes [B] no 
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One pair of parentheses may lie 


within another. For 


example, 


what do you think is 


the correct common name 


for 


(((3 + 4) X 5) - 1) 


X 


2 ? 




[A] 


33 




[C] 


68 


[B] 


60 




[D] 


None of these 
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Example : 



Solve the equation 6y+ 12= 2y + (-8), "writing your work 
carefully. Then choose the response that describes your me- 



thod and 


check the completed 


solution. 




[A] 


I began by adding 


(-12) 


to both 


sides. 


[B] 


I began by adding 


(-12) 


+ (-2y) 


to both sides 


[c] 


I began by adding 


(-2y) 


to both 


sides. 
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Example 



42 The least common denominator we want is the least common 
multiple of 4, 10, 45, and 6, which is 

[A] 4 x 10 x 45 x 6 

[B] 2x3x5 

[C] 2X2X3X3X5 



:The - correct choice 
you : wfe.re unable to 
53 j* otherwise, go 
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Example : 
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If a number is divisible by 6 and 3, can we conclude that 
it is divisible by l8? 

[A] yes [B] no 
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Example : 

In Chapter 12 we first studied quadratic polynomials, that is, polynomials 
in one variable which are of degree two. Remember that in order to find the de- 
gree of a polynomial in one variable we first reduce it to common polynomial 
form. The degree is the highest power of the variable that occurs in any term 
when the polynomial is in common form. 



Which of the following are not quadratic polynomials over the 
real numbers? 



(I) 


4 - 2x - x 2 




(L) 


x(3x + 2) 2 


(J) 


(3 2 )x - 4 




(M) 


(2x + l) 2 + 


(K) 


(2 - x)(2x + 


1) 


(N) 


1 2 
¥ x 


[A] 


L 


[C] 


J, L, M 




[B] 


J, N 


[D] 


K, M, N 





Although' it -InvoXvos;^ 0 '.;s 4 uare of' 3, (tf) ; does hot involve 
the square of the varihbih hud is therefore not a quadratic 
polynomial'* . ;}/■/:■ ‘ V • '• 

If you phfc (l) r ih r £Op^k;.^ form, you* will -See; that •' 

it is of the third; #ho monomial, involving, the- hijgh- . 

est pOwer of x j , '•* } ; ■■ . •••••' .. ' ; > 

(H) is a poiynoMal of -degree- 1; Vefify this 1 bjrperfoi&ing 
the indicated squaring Operation and collecting. like terms v . ; 

£cl is the correct answer. .. 
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Example (confirmation of response followed by further development in text proper;: 

Yes, the set { -7, -2, 0, 1, 4, q, r, s} contains additive inverses of all the 
elements of set A = { -4, -1, 0, 2, 7). The set you have chosen also contains 
the elements q, r, and s. Do you think that a, r, and s might also be 
inverse of some of the elements in set A? 

We know that sometimes there is more than one number which will satisfy a 
given condition. For example, both 3 and (- 3 ) have an absolute value of 3- 
We are now interested in finding out whether there is more than one inverse for 

a given number. 

Let us take the number 3 and assume that an additive inverse of 3 1S 
the number z. Then we can write: 3 + z = 0. One value of which makes this 

open sentence true is -3- 



Based on what we have studied, is there another value of z 
which makes 3 + z = 0 a true sentence? 

[A] yes [B] no [C] I don’t know 







\~ - . et- ■.■exists^;"; -Cqntih^'4 4£L • 
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arid. the. aefinx^iop- o^' the inverse, to prove that- there, , ' 

is ohly one additive inverse -foreach numbei%, /L^t/us^y^il^. it f^st .for ' 
a particular number./ 3v.X ■ ■ ;‘ - ' P / 

P7ith these examples, it should he clear that frames vary in size and purpose. 
Some contain cues to the response sought while others depend on recall or paral- 
lelism to other material. The number of responses within a frame may vary con- 
siderably, too. Some frames are used to check understandings, others to anti- 
cipate new material. 

There was a tendency in earlier programing efforts to fragment material for 
the sake of breaking it into small steps. The assumption that behavior was 
shaped in this way led many of us to sterilize our language and avoid coherence 
lest it make the step too great for a student. Only as we began to see proofs 
failing for lack of direction, exercises performed mechanically without a pattern 
forming, and page after page of "dull, boring, trivia", did we realize that mathe- 
matics should not be programed restrictively in frames of three lines’ length with 
one response per frame. In the 1962 version of the SMSG Programed First Course 
_in Algebra Form CR, (p. 6-7) we find, the following frames: 



50 

51 

52 

53 

5^ 
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There are four whole numbers between 5 and 10, 
namely the numbers 6, 7, 8, and 9. There are 
whole numbers between 5 and. 9? 



how many 
There are 



how many 



There are 



whole numbers between 5 and 8? 
whole numbers between 5 and 7? 



how many 

There are whole numbers between 5 and. 6? 

no, some 

The set of even numbers which are greater than 5 and 
also less than 9 contains element(s). 

Finally. . . 

The set which contains no elements is called "the empty 
set" (or "the null set"). Since the set of whole num- 
bers between 5 and. 6 does not have any elements in 
it, it is the set. 



: 3 
2 
.1 
no: 

2 . 



'/empty - 



m 






o 
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As experiences in -writing was gained, frames grew larger, incorporating 
examples, proofs, or topics within and often requiring more than one response. 
Some dependence on earlier material may he noted in the following pair of 
frames : 



Earlier in this course we solved equations such as 
x 2 + 6x + 5 = 0. It would he interesting to see 
what relations exist between the set of 

p 2 

x + 6x + 5 = 0 and the graph of y = x + 6x + 5. 
with reference to the same set of axes, draw the 
graphs of: 

y = x 2 + 6x + 5 
y = x 2 + 6x + 9 

y - x 2 + 6x + 13 



Notice the parabolas 



2 

y = x + 6x + 5 

p 

y = x + 6x + 9 
y = x 2 + 6x + 13 



(1) All have the same shape and. the same axis 

of . 

(2) However, their vertices are (-3; -*0 } 

(-3, ) , and. ( , ) respectively. 

p 

The curve y = x + 6x + appears to intersect 
the horizontal axis in the two points ( , 0) 

and. ( , 0) . 

p 

The curve y = x + 6x + 9 appears to intersect 
the horizontal axis in the single point ( , ) . 

2 

The curve y = x + 6x + 13 

(does, does not) 
intersect the horizontal axis. 






f; sfeh. r^spo'nee # 

:■ • 'iey page 




.symmetry • = 

. ' . -C; 

: (- 3 , 0 ), (- 3 ># 

{•1, 0) 

(-3, 0) • ; ; 

'does not. 
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Just as there was a tendency to fragment the content, so’ too there was a 
tendency to force the same response over and over again. The notion of rein- 
forcement was somehow confused with use, repetition, and variety of presentation. 

But this is a far cry from the tendency to claim "it is obvious that...” or "it 
follows simply. . . " 

Periodically, programers, teachers, and students expressed a need for items 
for which the responses were not immediately confirmed. In the summer of 1963 , 
review sections of the course were presented with problem sets having answers in 
a key at the back of the book. In addition, certain optional frames and other 
items where a check point or criterion item seems necessary, were programed in 
this way. Consider the following: 



52 

53 

54 

55 



Solve the following equations by squaring. The answers are on 
page xx. If when you have finished you find all of your answers 
are correct, go on to Item 76 ; if you have difficulty or wish 
to check your work, go on with Items 57-76. 

V 2 x = 1 + x 

V 2 x + 1 = x + 1 

■^ x+ l - 1 = x Hint: write first as i/x +1 = x + 1 

7x - 1 = x - 7 



In addition to giving hints and complete solutions, it is possible for a 
program to lead a student into depth outside the text he- is using. Often a 
teacher will provide some students in a class with extra work for enrichment; 
so then the program can tell the student: "If you had no difficulty with this 

and would be interested in other cases, read the following pages in..." . An 
inexpensive booklist is available from SMSG in Newsletter 17 from which ref- 
erences can be taken. In the 1964 revision of Form H, a topical reference 

guide to the New Mathematical Library series has been incorporated within the 
text. 

Check points are often needed to be sure that students are performing on 
test- like questions as well as they are performing on individual frames. Nhile 
this is frequently done by short quizzes in class, the programed text can' build 
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these in as a series of self-tests. In the summer of 1963^ s. self-test was 
written in Chapter 12 (Section 6b) with answers in the back of the text and 
references to the earlier section on which the question depends. The student 
can thus return to topics with which he had. trouble or which he failed, to mas- 
ter the first time through. 

There are many ways in which students can be cued or prompted to give the 
correct response to an item. Although out of context they seem too obvious , 
they are frequently used to elicit a student response early in the program. 

Underscoring - often used the first time a new word is used or to emphasize 
a word or phrase; e.g. 



A startling development occurs if we consider an 
original set which is an infinite set . 

N = {1, 2, 3, 4,...} is an 

E = {2, k, 6, 8, ...} is also an 

and. it is also a proper of N. 



infinite 'set ., 
infinite set : 
Subset 



Response blank cue - used when we provide a student with information on 
style of response, when only a part of the response has been worked on, or when 
data is being accumulated, e.g. 



I — 

Solve the inequality: x+ (-2) >-3* 

(x + (-2)) + 2 > -3 + 
and x + ((-2) + 2) = x + 

so x > 



-3-+ •• 

0 \ ; 

x"> “1 • 



Note that the student is given part of the response desired, often to make 
mathematical sense, as in the following 



1 1 _ 1 
a ‘ b Q 
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an 



Which would he improved hy subsequently calling for ^ since the reciprocal 
of ab is what is desired, not ab. 

Fading cue - gradual reduction of the stimulus to elicit a given response 
(spelling is a good example but problem solving can also be done in this way); 

e.g. 



13 x 13 is 


169. 


Therefore 13 is a sq r — 


of 169. 






15 x 15 is 


225 . 


Therefore 15 is a 


of 225 . 







square root 
square root 



Choice cue - under the blank a choice may be given to a student to limit 
his choice to one of two or three words or symbols; e.g. 



If 2 is not a factor of 


y, then 


y is 


• 

(even, odd) 




or 






If x is to the left of 
then x y. 

(<, = 7 >) 


y on the 


number 


line, 



odd 



x <y 



Parallel cue - in the fading cue above, the items are written in parallel 
style; that is, the wording is similar from one item to the next. The following 
examples are common: 



Just as x + 3 is the same as «3 + x, so too, 
a + 5 is the same as . 



5 + a 



0 is a 


subset of 


{ 0 } 


True or False? 


0 is a 


of 


{ 0 , 1 } 





True 

subset 















mm 



Empirical cues - exercises in -which the student counts, reads, or performs 
some act to enable him to respond correctly; for example, 



If A = {0, 1, 2}, then A is not empty; 

tains elements. 

(how many) 

If B = {0, 1}, then B contains 

If C = {6}, then C contains 

If D = {0}, then D contains 



con- 



elements , 



elements, 
elements . 



Since D = {0} , D is not the empty set. 



2 

1 

1 



or 



See if you can complete the following table and discover a rule 
for finding the number of subsets in any finite set: 



number of 
elements 


1 


2 


3 


h 


5 


6 


• • • 


n 


number of 
subsets 


2 


k 


8 








• • • 




number of 
subsets in 
exponent form* 


2 1 


2 2 


2 3 








• • • 





1 2 

* also may be written in expanded form 2=2, 2=2*2, etc, 

When you finish, compare your table with that shown on page i. 



or 



Take a penny, a nickel, a dime, a quarter, and a half-dollar. Each has a 
head and tail. The number of heads and tails which can appear when the coins, 
or any number of coins, are on the table below. 

Let's record, some data. 

1. Use one coin. Notice we can have one head (H) or no head. 

2. Use two coins. Notice we can now have two heads, head or, no head. 

6 7 
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heads, 



4 . 

5 - 



Use three coins. Wow we can have _ 
or head • 

Use four coins. Record your findings below. 
Try five coins. 



heads, 



head, 



Number of 
coins 


Number of heads possible 


Number of tails possible 


1 


1 0 


0 1 


2 


2 0 


0 2 


3 




0 1 3 


4 


— 




5 





1 



With the coins above we find it possible to record the data in another way. 

2 coins 

3 coins 



4 coins 



HH 


HT 


TT 






TH 






HHH 


HHT 


HTT 


TTT 




HTH 


THT 






THH 


TTH 




HHHH 


HHHT 


HHTT 


HTTT 



TTTT 



Other types of cues and prompts, depending for the most parrt on the ability 
of the writer, include: 

1) visual cues, use of second color, italic or other typographical vari- 
ations (capital letters, bold face, etc.), d i agrams with accented lines 

2) conceptual cues, similarities, contrasts, references to the whole topic, 
analogy 

3) mechanical cues, verbal patterns, rote responses, word pairs, etc. 
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The term "panel” was introduced into the Jargon of programed learning when 
referring to material which was apart (like an appendix) from the program it self , 
as, for example, tables of statis+'cs, charts, pages of text, summaries, etc. 

One of the characteristics of the SMSG efforts has been to develop panels con- 
structed in part by the student. The use of the constructed panel is varied. 

It can serve as:, a summary of properties, for example, which are studied through- 
out the chapter and recorded by the student as he completes each one. The panel 
may record and preserve the totality of a proof on which the student is working. 
The panel may summarize experimental data gathered to which the student may 
later be referred. The panel may preserve the equation as a first step, the 
table of values as a second step, and the graph as a third step. A panel may 
simply display information or illustrations which are too bulky to place in the 
text. 



Let us consider a few examples of pupil-constructed panels: 





PANEL 


I 




1. 


S = {0, 5, 7 , 9 ) 


1 1 1 1 1 


| M 1 1 1 


2. 


T = { 0 , 2 , 4 , 6, 8, 10 ) 


0 II 1 1 
II- 1111 


II 1 1 1 10 
M II 1 1 


3 - 




0 I I II 1 
l.l Mil 


r 1 1 1 1 io 
M 1 1 1 1 


4 . 




0 1 1 1 1 1 
1 1 1 1 1 1 


1 1 1 1 io 
1 | | | 1 


5 . 


M = {0, lj 3, 5 > ♦ • • } 


o 1 1 1 1 1 
_L i i i i i 


1 1 I M00 

||||l 






o 1 l 1 1 1 


' * ' MO 






> pbuw jj *,« .u*j ^ — --- i - ' — 



. j 
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Clearly f A] is the graph of the set {l, 3, 5 , ?, 9} » is this 
" the set Si [B] is the graph of K # [C] is the graph of set 
SZ If . you were correct, try graphing sets T, K f .'V, anci Mr 

Notice that the first reference to the panel makes sure that the student 
has the correct panel. If the panel is on the response sheet it may he used in 
the following manner: 

* 

Suppose a student is to complete a proof on his own. He could he told in 
the text that he should work on the Panel until he needs help, then refer to the 
item which gave him trouble. If a less able student does not work through the 
Panel alone, he may do so with guidance from the text. The following example 
illustrates the interplay between text and response sheet (on which the panel is 
located) : 
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Try to complete the proof of this theorem for yourself. On your response 
sheet is an outline of a proof (see Panel 7-2). Do not refer to the items be- 
low unless you find, blanks in the proof which you cannot complete. In such a 
case, refer to the item below which is numbered to correspond, to the one which 
gave you some trouble. 



22 

23 



THEOREM: If a is any real number, then a-1 = a and l«a = a. 



The theorem we are about to prove deals with the 
property of 1. We have already shown that this property 
holds for all numbers. 



Kmul^tlpiication ' 



* s’-' ' > ' * 




2b 
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CASE I assumes then that a is a number ^ 

for which the property of 1 is known to 

hold. Here, there is no problem^ when a is nonnegative, 
a»l = a. 



1 ;^niiegative. : - "/= 

f s ** * *.•*:. * . ! 

..t;^tipiication'' 



26 

27 

28 

29 



30 

31 

32 

33 

3^ 



Case II assumes that a is a number. To 

prove that this property still holds, we go back to the 

of multiplication for real numbers. This 

definition states (in part) that if one of the numbers is 
and the other is } then 

ab = -( |a| • |b| ) . 

We know that |l| = 1 by the definition of absolute value, 
and we know that in this case a is negative, so 

a *l = -( | aj *1). 

|a| is nonnegative so |a|*l = . The opposite of 

(|a|.l) is ; so too the opposite of |a| is 

Since a is negative and |a| is then the opposite of 

a, we can write |a| = . Further, the opposite of 

the opposite of a real number is the number itself, so 
— |a| is another name for -(-a) when is negative. 

We have thus shown that when a is negative, -|a| = 

-(-a) = a. 



• negative 



' definition. 

*. * * ■"■<> 

*honnegaiive,: : , 
=•’ 'negative''.} 




~a. 

* / 

& 
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The proof is now complete because each real number is either nonnegative or 
negative; in both cases above, we have shown that a*l = a. Check your completed 
proof with the one given on the next page. Note that there may be more than one 
proof for the theorem, but that the one given is sufficient to demonstrate the 
truth of the theorem. 



PANEL 7-2 • 

THEOREM: If a is any real number, then a«l = a. 

PROOF: 

CASE I: Assume that a is nonnegative 

Then, if a is nonnegative, a«l = a because the 

^ property of 1 is known to be true for all 

23 real numbers. 

CASE II: Assume that a is negative. 

26 Since in this case a is and 1 is 

27 nonnegative, a»l = — ( | a | • 1 1 1 ) by the of multiplication 

of real numbers; since |l| = 1 by the definition of absolute value, 
we can write 

a*l = -( |a| • |l( ) 

= -(|a|.l) 

and use the multiplication property of 1 for nonnegative real 
numbers to write 

32 - ( J a | • l) = - | a | . 

33 But, since a is negative, |a| = by definition of 

34 absolute value; hence, -|a| = -(-a) or because the opposite 

of the opposite of a real number is the number itself. 

In both cases, we have shown that a*l = a is 

(true, false) 

when a is any real number. 

Check your completed proof with the one given on page xx. 












2-5 Revision : Comments and Criticisms 

So you thought your first draft was good, eh? Well, a mathematician, a 
teacher of high school mathematics, a psychologist, and a high school student 
will all he willing to read, your prose and. offer suggestions. They may, as 
some have, say, "You forgot a comma in line 3, a semicolon before "therefore" 
in line 7 s^nd misspelled "hypotenuse" on page two." The "fly-speckers" will 
find most of your typographical and grammatical errors on a first or second 
reading and. save you from the critics in the published reviews later. Don't 
underrate them! 

You may find your prose being suggested for the booby prize, since you 
failed, to specify the domain of the variable in the equations on page 2, and 
again on page 3 where failure to specify the domain allows the equation to 
become either indeterminate or meaningless, depending on the value chosen." 

The same critic may call your writing "mathematical nonsense". The critics 
should be willing to be brutally frank, and you must be prepared, to hear their 
complaints. You should plan to do the same for others. 

It is possible that the teacher-reviewer will claim that "if the wording 
were simplified, my ninth graders could, understand it. Why not use a few more 
concrete cases like those on page of the text?" 

The psychologist may offer criticism when he understands the mathematics. 

He may suggest that "the stud.ent does not seem to be prepared for this particu- 
lar notion.' Other comments might be: "Does this response .really add to under- 

standing of the problem?" "Why repeat this question as often?" "Wouldn't it 
be better to schedule the item in later for review purposes?" "The shift to 
this new topic is abrupt, how about some transition material?" 

"The topic is well- developed and logically presented but could, use another 
illustrative example before going on." Why not say, "For all real numbers, a, 
b, and c, (a + b) c = ac + be or c(a+b) = ca + cb, since the 'or* is valid 
whether the system is commutative or not. Right and. left-handedness should not 
concern the student at the elementary level." "Do you want the student to prove 
this or follow your proof?" "I would guess that at the rate of 60 items per 
hour, the program would last 250 hours. There are only 150 usable hours in 
a school year. Is this reasonable?" "Mcst students have trouble here. More 
drill is necessary." "Why belabor this?" 
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From the student, or an equally naive subject, you may find, that "the word- 
ing was ambiguous. I thought of at least five ways to complete the sentence 

p 

correctly." (x + 3x + 2 is .) Or, "Why did. you include so many examples 

on the easy topics and so few on the difficult ones?" 

2-6 Revision and, the Second. Draft 

The critics of the first draft are clearly not supposed to be gentle. If 
they are, your criticism later may be brutal. What use should authors make of 
criticism? 

In some cases, the comments require simply correction of typographical errors 
and. minor changes. These are easily done by persons expert in English but not 
necessarily expert in mathematics. But beware— sometimes a minor change of word- 
ing may change the meaning considerably. Recall R.E.K. Rourke’s experience on the 
Report of the Commission on Mathematics (CEEB) when he returned from Russia to 
find that the editor had. found, "real" too redundant so had. changed some of the 
occurrences to "genuine" numbers. The change from "into" to "onto" could also 
be nontrivial. 

A goodly number of the comments are likely to be very general and. force the 
revision of several items and/or frames. Some will deal with clarity of presen- 
tation, some with length of sentences and. paragraphs, and. some with the mathematics 
per se. 

One of the findings which bears notice here is that in attempting to write 
mathematics for a self-instruction text forces the author to think more critically 
than ever before. The programed text, which may or may not work, brings out the 
good and the bad. in details of the mathematics and denies authors the glibness 
which they assume in conventional writing. In this important way, programing is 
as good, for the authors as it is for the students. Authors of programed texts 
must be prepared for the possibility that the mathematics, when approached from 
a learner’s point of view, may be different in structure from the mathematics 
when seen by the mathematician. 

The following questions may be asked in the course of the writing. The 
answers will vary considerably but cognizance of their presence may preclude 
some of the common mistakes made by many programers. 

1. What content is being taught? What kinds of student behavior are de- 
sired as end products? What student behavior exists at the beginning 











of the program? In what way is the content best presented? Is pro- 
graming the material an efficient expenditure of energy? 

2 . How long would a conventionally taught student spend on this material? 
Is the program to do the whole job of teaching? Are teachers films, 
or tapes available to the student? Does the program utilize non-text 
materials as resources? 

3 . Are the frames too long? Is a frame filled, with too much detail? Are 
the responses relevant to the skill, vocabulary, or concept being de- 
veloped.? If a mistake is anticipated, are corrective tracks provided? 
Are items poorly worded so that students will err? Are mistakes 
capitalized on by programers? 

4. Can the response be made without reading the entire frame? Is the 
response over-cued? Is the language appropriate to the student level? 
Is the wording artificial? Is the response repeated very often? Has 
a verbal pattern been fixed by the repeated use of an expression? Are 
the responses accumulating to the total behavior desired? Can the 
student see where he is going? Are there too many frames in the unit? 
Are panels used effectively? for summary? for exploration? Can the 
student move into and. out of the panels effectively? 

5. Important . Did. the programer list the techniques which seem to work 
some of the time as a guide to future writing? 

The second, draft gives the programer a chance to work on one very important 
factor; style. The use of illustrative material and. consistency within frames 
can be improved, at this point. The programers should, be able to read, all ante- 
cedent sections and. make transitions smooth. 

Editing a programed text is not like editing a conventional text. The 
editor cannot scan, omit pages to be done later, change punctuation and/or word- 
ing, or rearrange passages randomly. In a program, the items and. frames are 
written for the purpose of shaping behavior and, insofar as they accomplish this 
end, they can withstand little revision. (That is not to say there* s only one way 
to shape behavior, but another path may be widely separate from the one chosen by 
the authors in a given program.) The editing of a programed text is necessarily 
almost continuous. Students, critics, and teachers may raise distinct questions. 

In the early days of the "art", error rate was considered, an important factor. 
The magic number (5 percent) seemed to prevail. Seldom did. authors raise the more 
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relevant issues: Are the mistakes trivial or significant? Does low error rate 

really say anything about the learning which is taking place? 



2-7 Student Try - out : Small and Large Scale Evaluation 

Bar-low (of Emory University) once called programed instruction, "Validated 
instruction". By this he meant simply that the material was validated by actual 
student trial before being unleashed on a large population. This notion has also 
been widely propagated by other psychologists who have maintained that "now we 
wxll truly get instruction at the student *s level." Beware i I The response 
elicited is in fact a sample of behavior and may not reflect knowledge of the 
whole. If a program is revised after one student try-out, we can expect another 
student to have made some different responses and. reactions. With two or three 
students we begin to find trends in response patterns and, after taking a sample 

of ten or fifteen, either the data reduction becomes a burden or the response 
pattern has stabilized. 



During the summer of 1963, a student was found to go through the first 
eight chapters of Form H. Problems involved: (l) he was above average ability; 

(2) he could work sections consecutively only if they were written and ready to 
read.; (3) he was motivated by remuneration and. the thought of skipping the 
course in his own school; (4) he could work as much as six hours in a day. 

Some programers have failed to note that there may be a cumulative effect 
of a student going through a program and. have reported different chapters checked 
on different groups of students (many who had not worked the previous chapters in 
programed form.) It may be less acceptable but more realistic to get experienced 
teachers to work on the materials and wait for a finished product to get evaluation 

Just as a single student provides learner data, a carefully designed expert 
ment can produce group data. The variables may include: 

1. grade level 

2. varieties of presentation such as 



a. self-paced 
t>. group-paced 

c. used as supplement to a text 

d. used as replacement for a text 
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3* variations of the program per se, such as 

a. size of step changes 

b. vocabulary changes 

c. response mode 

d. order of topics being presented. 

Large populations can be found in public and private schools on which experi- 
mentation can be performed. Teachers vary from receptive to antagonistic and 
influence their students accordingly. The large scale experiment involves a 
problem of data reduction on a grand scale. To undertake it without adequate 
preparation is suicidal. 

3-0 Program Analysis 

When is a program a "good" program? A variety of answers have been pro- 
posed in the past few years and others remain open to further investigation. 

In the next few sections, presentation of some techniques for analysis of pro- 
grams by those whose concern is the production of a ''good* 1 program will be 
attempted. Bear in mind that the criteria here are not designed to evaluate 
content, but formj not use, but construction. 

3-1 Criterion Items 

Early in the movement, psychologists and some programers maintained that it 
would be possible to isolate criterion frames or items and to measure the pro- 
gram in terms of achievement on those items. The questions raised by this tech- 
nique hinge on two basic issues: The validity of the question itself and the 

role of the question in the shaping of final behavior. Clearly a question may 
fail to test what it purports to test or may fail to discriminate among students 
of widely diverse ability. Further, one can ask which items are criterion items 
along the path and fail to see the total behavior shaping up. 

One of the characteristics of contemporary mathematics instruction is the 
spiral nature of the subject. The number line may occur in the very first chap- 
ter and be woven through the fabric of the course as a unifying thread. Should 
a "thorough understanding" (whatever that means) of the distributive property be 
required when the student is just beginning to study mathematics? Or should 
synthetic proof be forced into false bloom in geometry before an intuitive grasp 
of the notion of proof is acquired? 
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The fact that criterion frames are not yet shown to he reliable guides to 
the revision of programed material should, not be taken to mean that criterion 
frames or items are not useful. In fact, teachers use criterion questions dur- 
ing the course of instruction to see if a student is grasping a given concept. 

The criterion item may demand recall, recognition, extrapolation, analysis, or 
any of the other behaviors which are sought en route to the terminal goals. 

Their value in programed instruction is not unlike their value in the classroom — 
they tend, to shape the instructor’s course of action. 

3-2 Flow Charts and. Concept Matrices 

In the writing of a program, it may help to have a flow chart or matrix 
showing the accumulation of ideas. Flow charts may be similar to the task analy- 
sis charts of Gagne or simply a listing of topics with their antecedents expli- 
citly noted. 

A more sophisticated format was suggested by the IBM Guide to Writing Programs. 
It was known as the "concept matrix", a rectangular matrix in which rows contain 
the concepts and columns contain item numbers. Within the matrix a cross indicates 
a definition, a dot represents a relationship containing the concept, and "C" 
represents a contrast item. In one sense, the concept matrix represents a flow 
chart to show the introduction, definition, relation, and. contrasts of newly de- 
veloped ideas. Such a scheme, if outlined before items are written could, be of 
assistance in fading schedules and determining critical points for the insertion 
of criterion items. 

3-3 logical Analysis 

In the programing of the SMSG First Course in Algebra it became clear that 
some topics were easier to program than others. Which ones? But more important 
— Why? Recall that this effort was essentially a translation from existing text. 

The content and structure of the text were preserved for the most part. Let’s 
look at the difficult spots. 

1. An early section on the number line. Why? What is assumed? What 
should, a student know when he has completed this section? What in- 
formation is to be included? rational numbers? successors? betweenness? 

2. The chapter on the multiplication of real numbers. Why? Is it parallel 
to addition? What cases can be proved? How much proof is appropriate? 

How long should the material take? 
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3. Manipulation with radicals. Why? What earlier material is needed? 
what exceptions should he pointed out? Are proofs necessary? What 
should the student he ahle to do when he has studied radicals? 

Polynomials. Why? What are they? Are we to consider their form a- 
part from their roles as functions? How shall we develop factorization? 

These are some of the trouble spots. What appears as we ask where trouble starts 
is a conjecture: Topics for which the subject matter does not force an explicit 

orderly development are hardest to program, to write, to teach, or to learn. In 
spite of the often quoted "Mathematics is a logical science," there appear to be 
aspects about which we know little at this time— among them, how students learn 

mathematics. Perhaps when we know more, we shall be able to shed the shackles 
and teach. 

3-^ Data: Collection and. Reduction 

To guide teachers as users and. programers as generators of materials, cer- 
tain data have been shown to be effective. To those undertaking the programing 
of mathematics, the task may seem monumental, but it bears good fruit. 

A series of tests were constructed to measure achievement in algebra. These 
included two pre-tests to see what background was on hand, and a series of chapter 
tests, and two post- tests to measure final' achievement. A standardized aptitude 
test was given to enable grouping into quartiles for analysis purposes. A sam- 
ple of 100 students were randomly selected from the population and their tests 
analyzed. The following data has been useful to the programers: 

1. On each item on a chapter test 

a. the percent of students scoring correctly on the item; 

b. the percent of students missing the item; 

c. the correlation of each item with the total score on the test; 

d. the number of students in each SCAT quartile who missed each item; 

e. a histogram showing the error frequency for each item; 

2. On the total test 

a. mean score; 

b. standard deviation; 

c. a histogram showing total score frequency; 
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On the ability test 

a. mean score; 

b. standard deviation; 

c. a histogram showing total score frequency; 
Teacher reactions 

Student opinions 



In the end, analysis of pr'e and post testing can be made as well as corre- 
lations of: ability measure with performance, specific items with pre and post 

testing, and other factors (such as reading) with specific sections or skills 
(such as problem solving). 

To facilitate data collection and reduction, forms can be devised for use 
by teachers and/or students. Tailor-made response sheets have been found to be 
more advantageous than a fixed response sheet format. Teachers, when loaded 
with a myriad of other teaching duties, are "reluctant" (at best) to engage in 
data accumulation. Students may be provided identification numbers at the start, 
classes or schools may be identified by number, rosters can be dittoed, students 
should be relied on to provide accurate data on their time and/or errors. 
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APPENDIX 1 - ADDENDA BASED ON 1964 EXPERIENCE 

Al-1 The Relation between Program and Text 

The original version of the programed First Course in Algebra followed close- 
ly the SMSG text First Course in Algebra in that the order of presentation, the 
examples, and the exercises were those of the text. As noted earlier, careful 
section by section analysis of content, behavioral goals, and. teaching sequence 
is the first phase of programing. Thus the availability of a well- designed text 
reduces the time required, for this phase. However, it is an over- simplification 
to regard the ideal program- -were one to exist — as simply a more elaborate version 
of a corresponding ideal text. 

A text is designed for use in a class, and the programer at certain points 
is programing the behavior of a teacher as much as the text itself. For example, 
a text may include exercises which lead, into the next topic to be covered. The 
purpose of such exercises is to motivate, and this purpose may be served, even 
though few students are able to solve the problems without help. Simply to pro- 
gram the solutions completely would diminish rather than enhance interest in 
what follows. 

Again, a text may use a rather complicated example, illustrating several 
ideas. The teacher can use such an example as the basis for class discussion, 
perhaps after students have read the material in the text. However, programing 
a lengthy example before background has been developed may create a bulky and 
difficult passage in which the main ideas are lost. 

A teacher is often expected, to supply ideas or interpretations not explicitly 
covered in a text. To help the teacher serve as a resource in this way, a com- 
mentary often supplies the teacher with information not given in detail in the 
text. Presumably, the teacher’s comments on a question or discussion relate the 
student’s ideas to more advanced or extended concepts. For the programer, the 
commentary presents a special sort of problem. It is desirable to convey to the 
student a sense of wider horizons without elaborating as fully as does the com- 
mentary. Thus occasionally, the program includes phrases such as "You will find, 
later that...", without requiring any explicit response from the student. 

In a spiral curriculum, it is not expected that an idea will be mastered on 
the student’s first encounter with it. Similarly, it is appropriate to present 
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material which the student is expected to read hut not necessarily to master 



fully. For example, in forming an idea of Proof, some students profit from 



engaging attention in such instances. 

If a text contains lengthy problem lists, teachers may assign only selected 
problems from them. The programer, on the other hand, must make a decision 
about the number of problems to be included. 

While observation or tests show that students have not mastered certain 
material wel!, teacher- critics tend to ask for more problems and drill. Some- 
times, however, it appears probable that a different organization and presenta- 
tion would be more helpful than additional drill. For example, the material on 
factoring polynomials caused great difficulty even though a first revision had 
incorporated more examples than had the original draft. It appeared possible 
that this topic is one where the teacher* s writing on the blackboard is important 
by providing a visual, moving stimulus. Thus, it is plausible that rearranging 
the presentation rather than adding problems is called for here. 

Even experienced teachers find that the experience of writing a program can 
be rewarding and provocative, though it is arduous. It calls for a fresh view 
of the elements involved in teaching. Likewise, it has been suggested that 
programed materials might be of value to inexperienced teachers in planning pre- 
sentations. For the new teacher, the program, which is far more detailed than a 
text, may serve as a helpful guide to systematic organization and to the estab- 



Al-2 Special Features of a "Hybrid. " Program 

As illustrated earlier , the later versions of the program, Forms H, con- 
tain multiple-choice and constructed response items as well as passages of straight 
text. Different teaching situations lend themselves more readily to one mode of 
instruction or another, and the programer using a "hybrid" form tends to develop a 
feeling that for any given purpose there is usually a mode that is particularly 
apt. Use of multiple-choice questions appears natural, for example, when one 
wishes to warn the student against a certain error, to discriminate between fine 
shades of meaning, or to remind the student of the reason that a certain response 
is incorrect. A multiple-choice question may also be used, as on page 57, in 
situations where several answers are appropriate, or in general, where comment on 
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i seeing proofs before they are able to construct them independently. The use of 
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1 panels with constructed response items referring directly to them, is a way of 




lishment of objectives. 
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alternative answers is called for. A constructed response form, on the other 
hand, often appears preferable for the step by step development of an idea. The 
programer sometimes varies the type of item simply to avoid, monotony. 

Teachers have commented that when students use programs exclusively, they 
may fail to develop habits of organizing written work in an orderly way. For 
this reason, the complete solutions to certain lists of problems, are given in 
the answer key at the back of the book. Students are not told that their work 
must duplicate the form of the solution given, but it is hoped that by comparing 
their work with acceptable models, they will become more proficient at presenting 
complete solutions. 

Almost every chapter ends with a summary which is not programed. The sum- 
mary (See APPENDIX 2) is intended to make the book useful for review as well as 
to encourage the student to organize the main ideas that have been included in a 
chapter. Aside from the use of nonprogramed. material to summarize developments 
that have been presented, and to offer relief from strict progr ami ng, expository 
sections are also used to introduce chapters by placing the material within 
specific frames of reference. 

The inclusion of nonprogramed material is expected, to make the program more 
adaptable to situations where its use is supplemented by teacher-directed, class- 
room activities. It appears that programs will be used in this way in some schools. 
For this reason, the teachers* commentary suggests to teachers appropriate tech- 
niques for the purpose. 

Some critics have called, attention to the danger that the techniques of pro- 
graming create the idea that each problem can be solved in only one way. To 
avoid such rigidity, alternative methods of solving a problem are often included. 
Multiple-choice questions with more than one acceptable answer are sometimes used, 
as in the example on page 57* In some cases, students are directed to try to 
solve a problem for themselves and. refer to the detailed solution given in the 
program only if they have difficulty. 

To provide for individual differences, ^various procedures may be used. For 
example, a theorem may be stated on the basis of several examples and the formal 
proof made optional. As an illustration, the following may be cited: 
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Items 37-4-1 suggest 

Theorem 12- 4c . For positive integers a, b, and. c, if a is 
factor of b, and a is a factor of (b + c), then a is 
a factor of c. 



*Jf2. If you wish, try to prove Theorem 1 2- 4 c and. then turn to 
page i to see one method of proving it. 



Problem lists may also include optional, more difficult examples for more 
able students. Moreover, some relatively simple problems lists for drill are 
incorporated in a way that permits skipping by more able students. Typical 
accompanying instructions are: "If you had difficulty with these Items, do 

Item t0 Ite/P * for further practice, " and. "If you answered Item 

correctly , proceed to Item . v This type o.f branching may be used for either 

constructed, response or multiple-choice items. 
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APPENDIX 2 - SAMPLE CHAPTER SUMMARY* 

CHAPTER 7 - Properties of Addition 

We have defined addition of real numbers as follows: 

The suih of two nonnegative numbers is familiar from arithmetic. 

The sum of two negative numbers is negative; the absolute value of this 
sum is the sum of the absolute values of the numbers. 






The sum of numbers, of which one is positive (or 0) and the other is nega- 
t±ve 9 is obtained as follows: 

The absolute value of the sum is the difference of the absolute values 
of the numbers. 

The sum is positive if the positive numbers has the greater absolute 
value . 

The sum is negative if the negative number has the greater absolute 
value . 

The sum is 0 if the positive and negative numbers have the same 
absolute value. 
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We have satisfied ourselves that the following properties hold for addition 
of real numbers: 

Commutative Property of Addition : For any two real numbers a and b, 

a + b = b + a. 

Associative Property of Addition : For any real numbers a, b, and c, 

(a + b) + c = a + (b + c). 

Addition Property of Opposites : For every real number a, ] 

a + (-a) = 0 and (-a) + a = 0. \ 

« I 

( 

* This is an example of chapter summaries in the student text (See page 84 of j 

Manual) and. is not illustrative of the chapter summaries and objectives prepared \ 

by Haag, Rickart, and Poliak (page 3 ) as 
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Addition Property of 0: For every real number a, 

a + 0 = a and 0 + a = a. 

We also stated, a fact, already clear from our earlier ideas, which we called, 
the addition property of equality : 

For any real numbers a, b, and. c, 
if a = b, then a + c = b + c and. c + a = c + b. 

This idea supplied, us with a useful procedure for determining the truth sets of 
open sentences. 

We have proved that the additive inverse is unique- -that is, that each number 
has exactly one additive inverse, which we call its opposite. 

We have discovered and. proved the fact that the opposite of the sum of two 
numbers is the same as the sum of their opposites. 
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APPENDIX 3 - SAMPLE SUMMARY OF CHAPTERS 

10-5 The Real Numbers - Summary 

Let us summarize the material of Chapters 6-10. 

We began "by introducing the set of negative numbers, as suggested "by our 
knowledge of the numbers of arithmetic and. the number line. We called the union 
of the set of negative numbers and the set of numbers of arithmetic the set of 
real numbers . 

Addition and multiplication of real numbers were then defined in such a 
way that 

1. the usual meanings of addition and multiplication in the set of numbers 
of arithmetic were preserved, and 

2. many of the properties of these operations, which were true for the set 
of numbers of arithmetic, were also true in the set of real numbers. 

We discussed an order relation in the set of real numbers, again based on 
our ideas of order in the set of numbers of arithmetic. 

In the course of this development we stated many properties of addition, 
multiplication, and order. 

Here is a list of certain basic properties which we considered: 





For any real numbers 


a, 


b, and c, 


1. 


a + b is a unique real number. 


1*. 


ab is a unique real number. 


2. 


a + b = b + a 


2*. 


ab = be. 


3- 


(a + b) + c = a + (b + c) 


3*. 


(ab)e = a(be) 


4. 


There is a special real number 


4». 


There is a special real number 




0 such that a + 0 = a and 


« 


1 such that a • 1 = a and 




0 + a = a. 




1 • a = a. 


5. 


For each a there is a unique 


5* • 


For each a, different from 0, 




real number (-a) such that 




there is a unique real number (— 

& 




a + (-a) = 0 and (-a) + a = 0 




such that 



a • — = 1 and — = 1. 
a a 
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(b + c)a = ba + ca . 



8. If a < b and. b < c, then a < c 



6. a(b + c) = ab + ac, 

7. For a and b, exactly one of the following is true: 



9. If a < b, then a + c < b + c 



a<b, a = b, b<a 




These are by no means all of the properties we discovered, but we will 
interrupt our listing at this point to make the following remark. If we add 
just one more basic property, we have a list of basic properties that could be 
used to prove everything about the real numbers. Unfortunately, this additional 
property would involve us in mathematics beyond the scope of this course. 

Practically all the algebra of this course may be developed from the four- 
teen properties listed, above. Let us now list some of the more important pro- 
perties which may be proved, as consequences of the basic properties. All of 
these have been discussed in the text. As you study more mathematics you will 
learn much more about the real numbers. 



(-a)(-b) = ab. 

7. The opposite of the opposite of a real number a is a. 

8. The reciprocal of the reciprocal of a non-zero real 
number a is a. 

_ . 1 1 1 

9. For any non- zero real numbers a and b, ^ * b = ab * 

10. If a and b are real numbers, then a < b, if and only if 

there is a positive number c such that b = a + c. 



1. For real numbers a, b, and c, if a+c=b+c 



then a = b 



2. For real numbers a, b, and c with c / 0, if 
ac = be, then a = b. 



3. For real numbers a and. b, ab = 0 if and only if 
a = 0 or b = 0. 



4. For any real number a, (-l)a = -a. 

5. For any real numbers a and. b, -(a + b) = (-a) + (-b) 

6. For any real numbers a and b, (-a)b = -(ab) and. 



i 



11. For any real numbers a, b, and c, 

if a < b and 0 < c, then 
ac < be, 

if a < b and c < 0, then 
be < ac. 



OJ 
1 — 1 


For 


any real numbers 


a and b, if a < b, then -b < -a. 


PO 
1 — 1 


If 


0 < a < b, then 


l<i 

b a 


• 

-^h 
1 — 1 


If 


/ 2 
x f 0, then x 


is positive. 


15 . 


If 


0 < a < b, then 


2 . .2 
a < b . 



j 

k 

3 




If we consider the set of real numbers together with the operations "+" 

and the relation and the basic properties we may call this the real 

number system. 
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APPENDIX 4 - SAMPLE OBJECTIVES 

...More detailed objectives of the course are given in the Preface to the 
Commentary for Teachers accompanying the First Course in Algebra. Any attempt 
to list objectives, however, tends to generalities. As an alternative, we list 
a set of sample questions each of which illustrates a particular objective of 
the course; in a parallel column we explain what the question is designed to 
test, or not test. There are three catagories of questions: language, structure, 

technique. It should be understood that these are sample questions and that no 
effort was made to include the type of routine questions that are needed in a 
good test. Many such routine questions can be found in the Suggested Test Items 
in the Commentary* 

Suggested Test Items in the Commentary 
Sample Questions for SMSG First Course in Algebra 

Language 

Item 

1. Another name for 2+3 • 4+1 
is 

(a) (2 + 3 ) • 4 + 1 

(b) (2 + 3) * (4 + 1) 

(c) 2 + (3 • 4) + 1 

(d) 2 + 3 • (4 + 1) 

(e) 2 + (3 • 4) + (3 • 1 ) 



Remarks 

There are certain accepted 
conventions concerning pre- 
ferred orders of operations. 
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Items 

Which of the following pairs of 
sentences have the same truth sets? 

I. 3(t - 4) + 5 * Mt + 1); 

3(x - 4) + 5 = 4(x + l) 

II. 2x + y = 1 and y = 2x + 1; 







t = 0 and S = 1 


III. 


x(x +4) « 

— = 2: x + 






X 


(a) 


only I 


0>) 


only II 


(c) 


only III 


(a) 


I 


and II 


(e) 


II 


and III 



12. If f is defined by 
f (x) = x + 1 
for all real x, then 
f(f(t)) is 

(a) (t + l) 2 

(b) x + t + 1 

(c) t(t + l) 

(d) 2t + 1 

(e) t + 2 



Remarks 



The equivalence of two sen- 
tences does not depend on the 
symbols used for the variables, 
but on their domains and the 
truth sets of the sentences. 

Here the domains must be implied 
from the sentences. 



Language of functions in- 
volves the notation and its 
use. 
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Structure 



Item 



22. Consider the set T of all num- 
bers of the form a + b 72 , where 

a , b, are any rational numbers. 

■Which of the following is not true? 

(a) the set T is closed under 
addition 

(b) the set T is closed under 
multiplication 

(c) 0 + 0/2 is the identity for 
addition in T 

(d.) every element in T has an 
additive inverse 

(e) 1 + 1-/2 is the identity for 

multiplication in T 



Remarks 

This is a difficult question, 
but it searches out the ideas 
involved in the structure of 
a field. 



Technique 

34. Which of the following is the 
graph of 

x 2 - 1 > 0? 



Skill in solving and graphing 
inequalities. 



(a) 


| 


L 


-i 


0>) 


-1 


1 

0 

I 


T 

1 

l 


(c) 


T 

-1 

i — 


1 

0 

— 1- 


T 

1 


(d) 


f 

-1 


1 

0 

1 


1 

1 - 


(e) < 


-1 


1 

0 

1 


T 

1 




-1 


I 

0 


1 
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